Transmission device, reception device, and communication system

ABSTRACT

A transmission device includes an image processing unit that controls transmission of a first packet including region information regarding a region detected from an image and controls transmission of a second packet for each of lines in which at least the region has been detected in the image. The second packet comprises a header including information regarding the line, and a payload including region data concerning a partial region corresponding to the line, out of the region, in which the header includes: identification information of the data included in the payload; an error correction code of the information included in the header; and an extension region provided so as to be interposed between the identification information and the error correction code, and information indicating whether or not the region data is included in the payload corresponding to the header is set in at least a part of the extension region.

FIELD

The present disclosure relates to a transmission device, a receptiondevice, and a communication system.

BACKGROUND

In recent years, there have been increasing opportunities to useapplications of transmitting a large amount of large-capacity data. Thisoften leads to application of heavy load to the transmission system, andin the worst case, the transmission system might be interrupted, leadingto a data transmission failure.

CITATION LIST Patent Literature

Patent Literature 1: JP 2016-201756 A

Patent Literature 2: JP 2013-164834 A

Patent Literature 3: JP 2012-209831 A

Patent Literature 4: JP 2014-39219 A

SUMMARY Technical Problem

Due to the above situation in the background, there has been a demandfor introducing a technology capable of reducing the load on thetransmission system, for example, by transmitting only the data of apartial region cut out from the captured image instead of transmittingthe captured image data as a whole. Note that techniques for cutting outa partial region from a captured image are disclosed in PatentLiteratures 1 to 4, for example.

In view of these, the present disclosure proposes a technology capableof transmitting data in a region that is set for an image in a morepreferable manner.

Solution to Problem

According to the present disclosure, a transmission device is providedthat includes an image processing unit that controls to transmit a firstpacket including region information regarding a region detected from animage and controls to transmit a second packet for each of lines inwhich at least the region has been detected in the image, the secondpacket including: a header including information regarding the line; anda payload including region data concerning a partial regioncorresponding to the line, out of the region, wherein the headerincludes: identification information of the data included in thepayload; an error correction code of the information included in theheader; and an extension region provided so as to be interposed betweenthe identification information and the error correction code, andinformation indicating whether or not the region data is included in thepayload corresponding to the header is set in at least a part of theextension region.

Moreover, according to the present disclosure, a reception device isprovided that includes an image processing unit that restores a partialimage corresponding to a region detected from an image, based on aresult of reception of a first packet including region informationregarding the region and based on a result of reception of a secondpacket for each of lines in which at least the region has been detectedin the image, the second packet including: a header includinginformation regarding the line; and a payload including region dataconcerning a partial region corresponding to the line, out of theregion, wherein the header includes: identification information of thedata included in the payload; an error correction code of theinformation included in the header; and an extension region provided soas to be interposed between the identification information and the errorcorrection code, and information indicating whether or not the regiondata is included in the payload corresponding to the header is set in atleast a part of the extension region.

According to the present disclosure, a communication system is providedthat includes: a transmission device including a first image processingunit that controls to transmit a first packet including regioninformation regarding a region detected from an image and controls totransmit a second packet for each of lines in which at least the regionhas been detected in the image, the second packet including a headerincluding information regarding the line, and a payload including regiondata concerning a partial region corresponding to the line, out of theregion; and a reception device including a second image processing unitthat restores a partial image corresponding to the region on the basisof a result of reception of the first packet and a result of receptionof the second packet for each of lines in which at least the region hasbeen detected in the image, wherein the header includes: identificationinformation of the data included in the payload; an error correctioncode of the information included in the header; and an extension regionprovided so as to be interposed between the identification informationand the error correction code, and information indicating whether or notthe region data is included in the payload corresponding to the headeris set in at least a part of the extension region.

Moreover, according to the present disclosure, a transmission device isprovided that includes an image processing unit that controls totransmit a first packet including region information regarding a regiondetected from an image and controls to transmit a second packet for eachof lines in which at least the region has been detected in the image,the second packet including: a header including information regardingthe line; and a payload including region data concerning a partialregion corresponding to the line, out of the region, wherein whether ornot the payload of the second packet includes the region data isrecognized on the receiving side of the second packet, based on at leastone of information included in the header corresponding to the payloador the region information.

Moreover, according to the present disclosure, a reception device isprovided that includes an image processing unit that restores a partialimage corresponding to a region detected from an image, based on aresult of reception of a first packet including region informationregarding the region and based on a result of reception of a secondpacket for each of lines in which at least the region has been detectedin the image, the second packet including: a header includinginformation regarding the line; and a payload including region dataconcerning a partial region corresponding to the line, out of theregion, wherein the image processing unit recognizes whether or not thepayload of the second packet includes the region data, based on at leastone of information included in the header corresponding to the payloador the region information.

Moreover, according to the present disclosure, a communication system isprovided that includes: a transmission device including a first imageprocessing unit that controls to transmit a first packet includingregion information regarding a region detected from an image andcontrols to transmit a second packet for each of lines in which at leastthe region has been detected in the image, the second packet including aheader including information regarding the line, and a payload includingregion data concerning a partial region corresponding to the line, outof the region; and a reception device including a second imageprocessing unit that restores a partial image corresponding to theregion on the basis of a result of reception of a first packet and aresult of reception of a second packet for each of lines in which atleast the region has been detected in the image, wherein the secondimage processing unit recognizes whether or not the payload of thesecond packet includes the region data, based on at least one ofinformation included in the header corresponding to the payload or theregion information.

Advantageous Effects of Invention

As described above, according to the present disclosure, there isprovided a technology capable of transmitting data in a region that isset for an image in a more preferable manner.

It is noted that the above effects are not necessarily limited, and,along with or instead of the above effects, any of the effects describedin the present specification or other effects which can be understoodfrom the present specification may be exhibited.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an example of a system configuration ofa communication system according to an embodiment of the presentdisclosure.

FIG. 2 is a diagram illustrating an example of a structure of a packetused for transmitting image data in the communication system accordingto the embodiment.

FIG. 3 is a diagram illustrating an extension region provided in aheader of a packet.

FIG. 4 is a diagram illustrating an extension region provided in aheader of a packet.

FIG. 5 is a diagram illustrating a format of data transmitted by a firsttransmission method.

FIG. 6 is a diagram illustrating an example of information included inEmbedded Data in the first transmission method.

FIG. 7 is a diagram illustrating an example of a configuration of apacket header in the first transmission method.

FIG. 8 is a diagram illustrating an example of a region that is set foran image.

FIG. 9 is a diagram illustrating an example of data transmitted by thefirst transmission method.

FIG. 10 is a diagram illustrating another example of a region that isset for an image.

FIG. 11 is a diagram illustrating another example of data transmitted bythe first transmission method.

FIG. 12 is a block diagram illustrating an example of a functionalconfiguration of an image sensor in the case of applying the firsttransmission method.

FIG. 13 is a block diagram illustrating an example of a functionalconfiguration of a processor in the case of applying the firsttransmission method.

FIG. 14 is a diagram illustrating a format of data transmitted by asecond transmission method.

FIG. 15 is a diagram illustrating an example of information included inEmbedded Data in the second transmission method.

FIG. 16 is a diagram illustrating an example of a configuration of apacket header in the second transmission method.

FIG. 17 is a diagram illustrating an example of data transmitted by thesecond transmission method.

FIG. 18 is a diagram illustrating another example of data transmitted bythe second transmission method.

FIG. 19 is a block diagram illustrating an example of a functionalconfiguration of an image sensor in the case of applying the secondtransmission method.

FIG. 20 is a block diagram illustrating an example of a functionalconfiguration of a processor in the case of applying the secondtransmission method.

FIG. 21 is a diagram illustrating a format of data transmitted by athird transmission method.

FIG. 22 is a diagram illustrating an example of information included inEmbedded Data in the third transmission method.

FIG. 23 is a diagram illustrating an example of a configuration of apacket header in the third transmission method.

FIG. 24 is a diagram illustrating an example of data transmitted by thethird transmission method.

FIG. 25 is a diagram illustrating another example of data transmitted bythe third transmission method.

FIG. 26 is a block diagram illustrating an example of a functionalconfiguration of an image sensor in the case of applying the thirdtransmission method.

FIG. 27 is a block diagram illustrating an example of a functionalconfiguration of a processor in the case of applying the thirdtransmission method.

FIG. 28 is a diagram illustrating a format of data transmitted by afourth transmission method.

FIG. 29 is a diagram illustrating an example of information included inEmbedded Data in the fourth transmission method.

FIG. 30 is a diagram illustrating an example of a configuration of apacket header in the fourth transmission method.

FIG. 31 is a diagram illustrating an example of information indicating aregion type.

FIG. 32 is a diagram illustrating an example of a region that is set foran image.

FIG. 33 is a diagram illustrating an example of data transmitted by thefourth transmission method.

FIG. 34 is a diagram illustrating another example of a region that isset for an image.

FIG. 35 is a diagram illustrating another example of data transmitted bythe fourth transmission method.

FIG. 36 is a block diagram illustrating an example of a functionalconfiguration of an image sensor in the case of applying the fourthtransmission method.

FIG. 37 is a block diagram illustrating an example of a functionalconfiguration of a processor in the case of applying the fourthtransmission method.

DESCRIPTION OF EMBODIMENTS

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings. Notethat in the present specification and drawings, components havingsubstantially the same functional configuration will be denoted by thesame reference numerals, and a redundant description thereof will beomitted.

Note that the description will be provided in the following order.

1. System configuration

2. Packet structure

3. Technical features

3.1. First transmission method

3.2. Second transmission method

3.3. Third transmission method

3.4. Fourth transmission method

3.5. Supplement

4. Conclusion

1. SYSTEM CONFIGURATION

First, a system configuration example of a communication systemaccording to an embodiment of the present disclosure will be described.

For example, FIG. 1 is a diagram illustrating an example of a systemconfiguration of a communication system according to an embodiment ofthe present disclosure.

A communication system 1000 includes an image sensor 100, a processor200, memory 800, and a display device 900, for example.

The image sensor 100 has an imaging function and a transmission functionand transmits data indicating an image generated by imaging. Theprocessor 200 receives the data transmitted from the image sensor 100and processes the received data. That is, in the communication system1000, the image sensor 100 functions as a transmission device and theprocessor 200 functions as a reception device.

Note that although FIG. 1 illustrates the communication system 1000having one image sensor 100, the number of image sensors 100 included inthe communication system according to the present embodiment is notlimited to the example illustrated in FIG. 1. For example, thecommunication system according to the present embodiment may include twoor more image sensors 100.

Note that although FIG. 1 illustrates the communication system 1000having one processor 200, the number of processor 200 included in thecommunication system according to the present embodiment is not limitedto the example illustrated in FIG. 1. For example, the communicationsystem according to the present embodiment may include two or moreprocessors 200.

In a communication system having a plurality of image sensors 100 andprocessors 200, the image sensors 100 and the processors 200 maycorrespond one-to-one, or one processor 200 may correspond to theplurality of image sensors 100. Furthermore, in a communication systemincluding a plurality of image sensors 100 and a plurality of processors200, the plurality of processors 200 may correspond to one image sensor100.

In a communication system including the plurality of image sensors 100and the plurality of processors 200, communication is performed betweenthe image sensors 100 and the processors 200 similarly to thecommunication system 1000 illustrated in FIG. 1.

The image sensor 100 and the processor 200 are electrically connected bya data bus B1. The data bus B1 is a signal transmission path thatconnects the image sensor 100 and the processor 200. For example, datarepresenting an image transmitted from the image sensor 100(hereinafter, sometimes referred to as “image data”) is transmitted fromthe image sensor 100 to the processor 200 via the data bus B1.

The image sensor 100 and the processor 200 may be electrically connectedby a control bus B2 different from the data bus B1, for example. Thecontrol bus B2 is a transmission path that connects the image sensor 100and the processor 200 for transmission of other signals. For example,control information output from the processor 200 may be transmittedfrom the processor 200 to the image sensor 100 via the control bus B2.

The control information may include information used for control and aprocessing command. Examples of the information used for control includepieces of information for controlling functions of the image sensor 100,such as an image size, a frame rate, and an amount of output delay fromthe reception of an image output command to the output of an image.Furthermore, the control information may include identificationinformation indicating the image sensor 100. An example of theidentification information is any type of information capable ofspecifying the image sensor 100, such as an ID set in the image sensor100.

Note that the information transmitted from the processor 200 to theimage sensor 100 via the control bus B2 is not limited to the aboveexample. For example, the processor 200 may transmit, via the controlbus B2, region designation information that designates a region of animage. An example of the region designation information is informationin any format capable of specifying the region, such as informationindicating the position of the pixel included in the region (forexample, coordinates indicating the position of the pixel included inthe region).

While FIG. 1 illustrates an example of electrically connecting the imagesensor 100 and the processor 200 by the control bus B2, the image sensor100 and the processor 200 need not necessarily be connected by thecontrol bus B2. For example, the image sensor 100 and the processor 200may transmit and receive control information or the like by wirelesscommunication using any transmission method.

Hereinafter, each of devices constituting the communication system 1000illustrated in FIG. 1 will be described.

(Memory 800)

The memory 800 is a recording medium included in the communicationsystem 1000. Examples of the memory 800 include volatile memory such asrandom access memory (RAM) and non-volatile memory such as flash memory.The memory 800 operates on electric power supplied from an internalpower supply (not illustrated) such as a battery included in thecommunication system 1000, or electric power supplied from an externalpower supply of the communication system 1000.

The memory 800 store an image output from the image sensor 100 forexample. Recording of an image to the memory 800 is controlled by theprocessor 200, for example.

(Display Device 900)

The display device 900 is a display device included in the communicationsystem 1000. Examples of the display device 900 include a liquid crystaldisplay and an organic electro-luminescence (EL) display (also referredto as an organic light emitting diode (OLED) display). The displaydevice 900 operates on electric power supplied from an internal powersupply (not illustrated) such as a battery included in the communicationsystem 1000, or electric power supplied from an external power supply ofthe communication system 1000.

The display screen of the display device 900 is used to display, forexample, various images and screens such as an image output from theimage sensor 100, a screen related to an application executed on theprocessor 200, and a screen related to a user interface (UI). Thedisplay of an image or the like on the display screen of the displaydevice 900 is controlled by the processor 200, for example.

(Processor 200 (Reception Device))

The processor 200 receives the data transmitted from the image sensor100 and processes the received data. As described above, the processor200 functions as a reception device in the communication system 1000. Anexample of a configuration related to processing of data transmittedfrom the image sensor 100 (configuration for functioning as a receptiondevice) will be described below.

The processor 200 includes one or more processors having an arithmeticcircuit such as a micro processing unit (MPU) and various processingcircuits. The processor 200 operates on electric power supplied from aninternal power supply (not illustrated) such as a battery included inthe communication system 1000, or electric power supplied from anexternal power supply of the communication system 1000.

The processor 200 performs various processes such as a process relatedto the recording control of image data to a recording medium such as thememory 800, a process related to the display control of an image onto adisplay screen of the display device 900, and a process of executingarbitrary application software. Examples of the process related torecording control include a “process of transferring control dataincluding a recording command and data to be recorded onto a recordingmedium to a recording medium such as the memory 800”. Furthermore,examples of the process related to the display control include “aprocess of transferring control data including a display command anddata to be displayed on the display screen to a display device such asthe display device 900”.

Furthermore, the processor 200 may control the function of the imagesensor 100 by transmitting control information to the image sensor 100,for example. The processor 200 can also control the data transmittedfrom the image sensor 100 by transmitting region designation informationto the image sensor 100, for example.

(Image Sensor 100 (Transmission Device))

The image sensor 100 has an imaging function and a transmission functionand transmits data indicating an image generated by imaging. Asdescribed above, the image sensor 100 functions as a transmission devicein the communication system 1000.

The image sensor 100 includes any type of image sensor device capable ofgenerating an image, such as “an imaging device such as a digital stillcamera, a digital video camera, or a stereo camera”, an “infraredsensor”, or a “distance image sensor”. The image sensor 100 has afunction of transmitting the generated image. The image generated by theimage sensor 100 corresponds to the data representing the sensing resultobtained by the image sensor 100. An example of the configuration of theimage sensor 100 will be described below.

The image sensor 100 transmits data (hereinafter, also referred to as“region data”) corresponding to a region that is set for an image by atransmission method according to each of embodiments described below.The control regarding the transmission of the region data is performedby a component (described below) that functions as an image processingunit in the image sensor 100. The region that is set for an image isreferred to as a region of interest (ROI) in some cases. In thefollowing, the region that is set for an image is referred to as eithera “region” or a “ROI”. That is, a term simply described as “region”indicates a region (ROI) that is set in an image unless otherwisespecified.

Examples of the process regarding the setting of a region for an imageinclude arbitrary processes capable of specifying a partial region in animage (or arbitrary processes capable of cutting out a partial regionfrom an image), such as “a process of detecting an object from an imageand setting a region including the detected object”, and “a process ofsetting a region designated by an operation onto an arbitrary operationdevice.

The process related to the setting of the region for the image may beperformed by either the image sensor 100 or by an external device suchas the processor 200. In a case where the image sensor 100 performs theprocess regarding the setting of the region for the image, the imagesensor 100 specifies a region according to the result of the processregarding the setting of the region for the image. Furthermore, forexample, in a case where the external device performs the processregarding the setting of the region for the image, the image sensor 100specifies the region based on the region designation informationacquired from the external device.

Transmission of the region data, that is, the data of a partial image bythe image sensor 100 enables reduction of the amount of data related tothe transmission compared with the case of transmitting an entire image.Accordingly, transmission of the region data by the image sensor 100leads to various effects achieved by smaller data amount, such asreduction of the transmission time, reduction of the load related to thetransmission in the communication system 1000, for example.

Note that the image sensor 100 can also transmit data representing anentire image.

In a case where the image sensor 100 has a function of transmittingregion data as well as a function of transmitting data representing theentire image, the image sensor 100 is capable of selectively switchingbetween transmission of region data and transmission of datarepresenting the entire image.

The image sensor 100 transmits either region data or data representingthe entire image, for example, according to the set operation mode. Theoperation mode is set by an operation on an arbitrary operation device,for example.

Furthermore, the image sensor 100 may selectively switch betweentransmission of region data and transmission of data representing theentire image, based on region designation information acquired from anexternal device. For example, the image sensor 100 transmits region dataof the region corresponding to the region designation information whenthe region designation information is acquired from the external device;or transmits the data representing an entire image when the regiondesignation information is not acquired from the external device.

The communication system 1000 has the configuration illustrated in FIG.1, for example. Note that the configuration of the communication systemaccording to the present embodiment is not limited to the exampleillustrated in FIG. 1.

For example, while FIG. 1 illustrates the image sensor 100 as an exampleof the device that functions as a transmission device, the device thatfunctions as the transmission device is not limited to the image sensor100. For example, in a case where the communication system according tothe present embodiment has a configuration including an image sensordevice such as an imaging device and a transmitter electricallyconnected to the image sensor device, the transmitter may function as atransmission device.

Furthermore, while FIG. 1 illustrates the processor 200 as an example ofthe device that functions as a reception device, the device thatfunctions as the reception device is not limited to the processor 200.For example, in the communication system according to the presentembodiment, any device having a function of receiving data can functionas the reception device.

Furthermore, in cases where an image transmitted from the image sensor100 is stored in a recording medium external to the communicationsystem, where an image transmitted from the image sensor 100 is storedin the memory included in the processor 200, or where the imagetransmitted from the image sensor 100 is not to be recorded, thecommunication system according to the present embodiment need notinclude the memory 800.

Furthermore, the communication system according to the presentembodiment can have a configuration not including the display device 900illustrated in FIG. 1.

Furthermore, the communication system according to the presentembodiment may have any configuration according to the function of anelectronic device to which the communication system according to thepresent embodiment described below is applied.

Application Examples

Subsequently, an application example of the communication systemaccording to the present embodiment will be described. Examples of thecommunication system 1000 include a communication device such as asmartphone, a drone (device capable of remote operation or autonomousoperation), and a moving body such as an automobile. Furthermore, theapplication example of the communication system 1000 is not limited tothe above example. That is, the present embodiment is applicable tovarious types of electronic devices such as a communication device suchas a smartphone, a drone (devices capable of remote operation orautonomous operation), a moving body such as an automobile, a computersuch as a personal computer (PC), a tablet devices and game machines.

2. PACKET STRUCTURE

Next, an example of the structure of a packet used for transmitting animage from the image sensor 100 (transmission device) to the processor200 (reception device) in the communication system according to thepresent embodiment will be described. In the communication systemaccording to the present embodiment, an image captured by the imagesensor 100 is divided into partial images in units of lines, and thedata of the partial image of each of the lines is transmitted using oneor more packets. This similarly applies to the region data concerningthe region that is set for the image (that is, image data of a portionfor which the ROI is set).

For example, FIG. 2 is a diagram illustrating an example of a structureof a packet used for transmitting image data in the communication systemaccording to the present embodiment. As illustrated in FIG. 2, a packet(Packet) used for image transmission is defined as a series of data thatstarts with a start code (Start Code) and ends with an end code (EndCode) in a data stream. Furthermore, the packet includes a header(Header) and payload data (Payload Data) arranged in this order. Afooter (Footer) may be added after the payload data. The payload data(hereinafter, also simply referred to as “payload”) includes pixel dataof partial images in units of lines. The header includes various typesof information regarding the line corresponding to the partial imageincluded in the payload. The footer contains additional (optional)information.

Here, the information included in the header will be described. Asillustrated in FIG. 2, the header includes items of “Frame Start”,“Frame End”, “Line Valid”, “Line Number”, “EBD Line”, “Data ID”,“Reserved”, and “Header ECC” in this order.

Frame Start is 1-bit information indicating the top of the frame. Forexample, Frame Start of the header of the packet used for transmittingthe pixel data of the first line out of the image data to be transmittedis set to value 1, while Frame Start of the header of the packet used totransmit the pixel data of another line is set to value 0. Note thatFrame Start corresponds to an example of “information indicating a startof a frame”.

Frame End is 1-bit information indicating the end of the frame. Forexample, Frame End of the header of the packet having a payloadcontaining pixel data at the ending line of the valid pixel region outof the image data to be transmitted is set to value 1, while Frame Endof the header of the packet used to transmit pixel data of another lineof the pixel data is set to value 0. The Frame End corresponds to anexample of “information indicating the end of the frame”.

Frame Start and Frame End correspond to an example of frame informationthat is information regarding a frame.

Line Valid is 1-bit information indicating whether or not the line ofpixel data stored in the payload is a line of valid pixels. Line Validof the header of the packet used for transmitting the pixel data of aline within a valid pixel region is set to value 1, while Line Valid ofthe header of the packet used for transmitting the pixel data of anotherline is set to value 0. Note that Line Valid corresponds to an exampleof “information indicating whether or not the corresponding line isvalid”.

Line Number is 13-bit information indicating the line number of the lineincluding the pixel data stored in the payload.

An EBD Line is 1-bit information indicating whether or not the lineincludes embedded data. That is, the EBD Line corresponds to an exampleof “information indicating whether or not the line includes embeddeddata”.

Data ID is 4-bit information for identifying each of data (that is, thedata included in the payload) when the data is transferred in aplurality of streams. Data ID corresponds to an example of“identification information of data included in payload”.

Line Valid, Line Number, EBD Line, and Data ID correspond to lineinformation that is information regarding a line.

“Reserved” is a 27-bit region for extension. In the following, theregion indicated as “Reserved” will also be referred to as an “extensionregion”. In addition, the entire header information has the data amountof 6 bytes.

As illustrated in FIG. 2, Header ECC that is arranged to follow theheader information includes a cyclic redundancy check (CRC) code that isa 2-byte error detection code calculated based on the 6-byte headerinformation. That is, the Header ECC corresponds to an example of “errorcorrection code of information included in header”. In addition, theHeader ECC includes the CRC code followed by two pieces of information,each of which is identical to the 8-byte information that is a set ofheader information and a CRC code.

That is, the header of one packet includes three sets of the identicalheader information and CRC code. The entire header has the total dataamount of 24 bytes, which is obtained by summing 8 bytes of the firstset of header information and CRC code, 8 bytes of the second set ofheader information and CRC code, and 8 bytes of the third set of headerinformation and CRC code.

Here, the extension region (Reserved) provided in the header of a packetwill be described with reference to FIGS. 3 and 4. FIGS. 3 and 4 arediagrams illustrating an extension region provided in the header of apacket.

As illustrated in FIG. 3, an extension region is a region to whichinformation indicating the type according to the information transmittedin the packet is set as the header information type (Header Info Type)for the top 3 bits. In accordance with the type of the headerinformation, the format of the information set in the remaining 24-bitregion of the extension region excluding the 3 bits for designating thetype of the header information (that is, the type of information and aposition at which the information is set) is determined. This allows thereceiving side to confirm the type of the header information torecognize what type of information is set at which position in theregion other than the region in the extension region designating thetype of the header information and to read out the information.

For example, FIG. 4 illustrates an example of setting the type of headerinformation and an example of the setting in a case where the payloadlength of a packet (in other words, line length) is set variable, as anexample method of using an extension region according to the setting.Specifically, the example illustrated in FIG. 4 sets a value accordingto the type in the case where the payload length is set variable, forthe type of header information. More specifically, the exampleillustrated in FIG. 4 sets “001” for the header information type, as avalue different from “000” set in the header information type in theexample illustrated in FIG. 3. That is, in this case, the typecorresponding to “001” out of the types of header information means thetype corresponding to the case where the payload length is variable.Furthermore, 14 bits in the extension region are assigned to “LineLength” in the example illustrated in FIG. 4. “Line Length” isinformation for notifying the payload length. This configuration allowsthe receiving side to recognize that the payload length is variablebased on the value set as the type of header information and read thevalue set as “Line Length” in the extension region, thereby enabling thereceiving side to recognize the payload length.

An example of the structure of a packet used for transmitting an imagefrom the image sensor 100 (transmission device) to the processor 200(reception device) in the communication system according to the presentembodiment has been described as above with reference to FIGS. 2 to 4.

3. TECHNICAL FEATURES

Next, as a technical feature of the communication system according tothe present disclosure, an example of a transmission method fortransmitting region data of a region (ROI) that is set for an image willbe individually described as a first transmission method to a fourthtransmission method.

<3.1. First Transmission Method>

First, a first transmission method will be described. The image sensor100 stores region data of the region set in the image into the payloadof the packet and transmits the stored data for each of lines.Accordingly, in the following description, a portion corresponding toeach of lines in the region set in the image is also referred to as a“partial region” for convenience.

(Data Format)

First, the format of data transmitted by the first transmission methodwill be described. For example, FIG. 5 is a diagram illustrating theformat of data transmitted by the first transmission method. In FIG. 5,a series of packets indicated by reference sign A1 schematicallyillustrates a packet used to transmit the region data concerning theregion set in the image (in other words, a packet used for transmittingthe data of the valid pixel region). The series of packets indicated byreference signs A2 and A3 correspond to packets different from thepacket used for transmitting the region data. In the followingdescription, when the packets indicated by reference signs A1, A2, andA3 are distinguished from each other, they are also referred to as“packet A1”, “packet A2”, and “packet A3” for convenience. That is,during the period in which data for one frame is transmitted, the seriesof packets A2 is transmitted before transmission of the series ofpackets A1. Furthermore, the series of packets A3 may be transmittedafter transmission of the series of packets. Note that at least one ofpackets A2 and A3 corresponds to an example of the “first packet”. Thepacket A1 corresponds to an example of the “second packet”.

In the example illustrated in FIG. 5, at least a part of the series ofpackets A2 is used for transmitting Embedded Data. For example, EmbeddedData may be transmitted in a state of being stored in the payload ofpacket A2. Furthermore, as another example, the embedded data may betransmitted in a state of being stored in a region other than thepayload of packet A2.

Embedded Data corresponds to additional information transmitted by theimage sensor 100 additionally (in other words, information embedded bythe image sensor 100), and this corresponds to information regardingimage capturing conditions or information regarding a region (ROI) forwhich region data is transmitted.

For example, FIG. 6 is a diagram illustrating an example of informationincluded in Embedded Data in the first transmission method. Asillustrated in FIG. 6, Embedded Data includes information such as “ROIID”, “upper left coordinate”, “height”, “width”, “AD word length (ADbit)”, “exposure”, “gain”, and sensing information, for example. Theexample illustrated in FIG. 6 also illustrates an example of the numberof bytes of each of pieces of information described above.

The information of “ROI ID”, “upper left coordinate”, “height”, and“width” corresponds to the information regarding the region (ROI) set inthe image and used for restoring the image of the region on thereceiving side. Specifically, the “ROI ID” is identification informationfor identifying each of regions. The “upper left coordinate” correspondsto the coordinate as an index of the position, within an image, of theregion set for the image and indicates upper left vertex coordinate in arectangular range in which the region is set. The “height” and the“width” indicate the height (length in the vertical direction) and width(length in the horizontal direction) of the rectangular region in whichthe region is set. Note that, among the embedded data, in particular,information regarding the region (ROI), such as the above-described “ROIID”, “upper left coordinate”, “height”, and “width” corresponds to anexample of “region information” included in the first packet (forexample, packet A2).

The “exposure” information indicates the exposure time regarding theimaging of the region (ROI). The “gain” information indicates the gainregarding the imaging of the region. The AD word length (AD bit)indicates the word length of data per pixel obtained by AD-conversion inthe region. Examples of the sensing information include the details ofcalculation regarding an object (subject) included in the region orsupplementary information for subsequent signal processing on the imagein the region.

In the example illustrated in FIG. 5, at least a part of packet A2 isused for transmitting Embedded Data. However, at least a part of packetA3, instead of packet A2, may be used for transmitting Embedded Data.Furthermore, in the following description, Embedded Data is alsoreferred to as “EBD”.

In FIG. 5, “SC” represents “Start Code”, which is a symbol groupindicating the start of a packet. The Start Code is added before thepacket. The Start Code is represented by, for example, four symbols ofK28.5, K27.7, K28.2, and K27.7, which are combinations of three types ofK Characters.

“EC” represents “End Code”, which is a symbol group indicating the endof a packet. The End Code is added after the packet. The End Code isrepresented by, for example, four symbols of K28.5, K27.7, K30.7, andK27.7, which are combinations of three types of K Characters.

“PH” represents a “packet header”, and corresponds to the headerdescribed with reference to FIG. 2, for example. “FS” represents a framestart (FS) packet. “FE” represents a frame end (FE) packet.

“DC” represents “Deskew Code”, which is a symbol group used forcorrecting Data Skew between lanes, that is, a deviation in receptiontiming of data received in individual lanes on the receiving side. TheDeskew Code is represented by four symbols of K28.5 and Any**, forexample.

“IC” indicates “Idle Code”, which is a symbol group repeatedlytransmitted during a period other than the transmission time of packetdata. The Idle Code is expressed in D00.0 (00000000) of the D Characterwhich is the 8B10B Code, for example.

“DATA” indicates region data stored in the payload (that is, pixel dataof a portion corresponding to the region set in the image).

“XY” corresponds to information indicating the position of the left end(the position in the image) of the partial region corresponding to theregion data stored in the payload as the X coordinate and the Ycoordinate. Hereinafter, the X coordinate and the Y coordinateindicating the position of the left end of the partial region, which isindicated by “XY”, will also be simply referred to as the “XY coordinateof the partial region”.

The XY coordinates of the partial region are stored at the top of thepayload of packet A1. In addition, in a case where there is no change inthe X coordinates of the corresponding partial regions betweenconsecutively transmitted packets A1 and the Y coordinate is incrementedby just 1, the XY coordinates of the partial regions may be omitted inpacket A1 to be transmitted later. Note that this control will bedescribed below separately with a specific example.

Furthermore, in the case of transmitting region data concerning apartial region corresponding to each of a plurality of region for a linein which a plurality of regions separated from each other in thehorizontal direction are set in the first transmission method, packet A1is generated individually for each of the plurality of regions andtransmitted. That is, two packets A1 are generated and transmitted for aline in which two regions separated from each other in the horizontaldirection are set.

Next, with reference to FIG. 7, an example of a configuration of thepacket header of packet A1 used for transmitting the region data of theregion (ROI) that is set in the image will be described, focusingparticularly on the configuration of the extension region. FIG. 7 is adiagram illustrating an example of a configuration of the packet headerin the first transmission method.

As illustrated in FIG. 7, in the case of transmitting the region data ofthe region (ROI) set in the image in the first transmission method,information indicating that the region information is to be transmitted(that is, the information corresponding to the type assuming thetransmission of the region information) is set as a header informationtype in the packet header of packet A1 used for transmitting the regiondata. Furthermore, information indicating that the region data (that is,region data concerning the partial region) is to be transmitted usingthe payload is set to at least a part of the extension region. Inaddition, in the case of transmitting the coordinates of the region(that is, the XY coordinates of the partial region) using the payload,information indicating that the coordinates of the region are to betransmitted is set for at least a part of the extension region. In thecase of transmitting the region data concerning the region (ROI) set inthe image, the payload length of packet A1 can change according to thewidth of the region in the horizontal direction. Therefore, informationindicating the payload length may be set in a part of the extensionregion, similarly to the example described with reference to FIG. 4.

Example of Data

Next, with reference to FIGS. 8 to 11, a specific example of datatransmitted by the first transmission method will be described as anexample in a case where the first transmission method is applied.

For example, FIG. 8 is a diagram illustrating an example of a region(ROI) that is set for an image. In FIG. 8, three regions of region 1,region 2, and region 3 are illustrated as an example of regions set foran image. Furthermore, in FIG. 8, a grid divided by broken linesextending in the vertical direction and the horizontal directionschematically illustrates a piece of unit data (for example, pixel)forming the image. In the example illustrated in FIG. 8, the coordinatesof the upper left vertex of each of grids are defined as the coordinatesof the grid for convenience.

Furthermore, FIG. 9 is a diagram illustrating an example of datatransmitted by the first transmission method, illustrating one-framedata in a case where the region data concerning each of the regionsillustrated in FIG. 8 is transmitted by the first transmission method.Note that the configurations of packets A2 and A3 in FIG. 9 aresubstantially similar to the example described with reference to FIG. 5,and hence the following description will focus on the configuration ofpacket A1. Furthermore, each of “1”, “2”, and “3” illustrated in FIG. 9corresponds to the data of region 1, the data of region 2, and the dataof region 3 stored in the payload of packet A1.

For example, the payload of packet A1 used for transmitting the data ofthe partial region in the first line of region 1 stores, at its top,coordinates (x,y)=(2,1) of the left end of the partial region, andstores, at its succeeding portion, region data concerning the partialregion. Moreover, in partial regions in the second and third lines ofregion 1 as a target of subsequent transmission of the region data, theX coordinates at the left end are similar to the partial region of thefirst line, with the Y coordinates being incremented by 1 from theimmediately preceding line. Therefore, in the payload of packet A1 usedfor transmitting the data of the partial regions on the second and thirdlines of region 1, the XY coordinates of the partial regions areomitted.

Meanwhile, in the lines corresponding to the third to fifth lines ofregion 1, the partial regions of region 1 and region 2 are set to beseparated in the horizontal direction. Accordingly, in transmission ofthese lines, packet A1 having the region data concerning the partialregion of region 1 stored in the payload and packet A1 having the regiondata of the partial region of region 2 stored in the payload arealternately transmitted. That is, in these lines, the correspondingpartial regions have mutually different X coordinates at the left endbetween packets A1 that are continuously transmitted. Therefore, forthese lines, the XY coordinates of the partial region (that is, thecoordinates (x, y) of the left end of the corresponding partial region)are stored at the top of the payload of each of packets A1.

Furthermore, region 3 is set to include three line regions in thehorizontal direction, in which the X coordinate of the left end of thepartial region corresponding to the first and third lines differs fromthe X coordinate of the left end of the partial region corresponding tothe second line. That is, in the lines corresponding to the first tothird lines of region 3, the corresponding partial regions have mutuallydifferent the X coordinate at the left end between packets A1 that arecontinuously transmitted. Therefore, for these lines, the XY coordinatesof the partial region (that is, the coordinates (x, y) of the left endof the corresponding partial region) are stored at the top of thepayload of each of packets A1.

Furthermore, FIG. 10 is a diagram illustrating another example of aregion (ROI) that is set for an image. In FIG. 10, three regions ofregion 1, region 2, and region 3 are illustrated as an example ofregions set for an image. Note that, in FIG. 10, a grid divided bybroken lines extending in the vertical direction and the horizontaldirection schematically illustrates a piece of unit data forming theimage, similarly to the example described with reference to FIG. 8.Therefore, also in the example illustrated in FIG. 10, the coordinatesof the upper left vertex of each of grids are defined as the coordinatesof the grid for convenience.

Furthermore, FIG. 11 is a diagram illustrating an example of datatransmitted by the first transmission method, illustrating one-framedata in a case where the region data concerning each of the regionsillustrated in FIG. 10 is transmitted by the first transmission method.Note that the configurations of packets A2 and A3 in FIG. 11 as well aresubstantially similar to the example described with reference to FIG. 5,and hence the description will focus on the configuration of packet A1.Furthermore, each of “1”, “2”, and “3” illustrated in FIG. 11corresponds to the data of region 1, the data of region 2, and the dataof region 3 stored in the payload of packet A1.

For example, the payload of packet A1 used for transmitting the data ofthe partial region in the first line of region 1 stores, at its top,coordinates (x,y)=(2,1) of the left end of the partial region, andstores, at its succeeding portion, region data concerning the partialregion. Moreover, in partial regions in the second to fifth lines ofregion 1 as a target of subsequent transmission of the region data, theX coordinates at the left end are similar to the partial region of thefirst line, with the Y coordinates being incremented by 1 from theimmediately preceding line. Therefore, in the payload of packet A1 usedfor transmitting the data of the partial regions on the second to fifthlines of region 1, the XY coordinates of the partial regions areomitted.

Furthermore, a portion on the right end side of the partial region onthe fifth line of region 1 overlaps a portion on the left end side ofthe partial region on the first line of region 2. Therefore, the payloadof packet A1 used for transmitting the data in the partial region in thefifth line of region 1 stores the region data concerning the partialregion in the first line of region 2 following the region dataconcerning the partial region in the fifth line in region 1. Note thatit is allowable to arbitrarily set whether the overlapping portionbetween the partial region of region 1 and the partial region of region2 is to be transmitted either as the region data concerning the partialregion of region 1 or the region data concerning the partial region ofregion 2. For example, in the example illustrated in FIG. 11, theoverlapping portion between the partial region of region 1 and thepartial region of region 2 is transmitted as region data concerning thepartial region of region 1.

Region 3 is similar to the example described with reference to FIGS. 8and 9, and thus detailed description thereof will be omitted.

Example of Configuration of Image Sensor 100 (Transmission Device)

Next, an example of a functional configuration of the image sensor 100(transmission device) in the case of applying the first transmissionmethod will be described. For example, FIG. 12 is a block diagramillustrating an example of a functional configuration of the imagesensor 100 in the case of applying the first transmission method. Asillustrated in FIG. 12, the image sensor 100 includes an image sensordevice 102 and an IC chip 104, for example. The image sensor 100 in FIG.12 operates on electric power supplied from an internal power supply(not illustrated) such as a battery included in the communication system1000, or electric power supplied from an external power supply of thecommunication system 1000.

Examples of the image sensor device 102 include image sensor devices ofany method capable of generating an image, such as “an imaging deviceincluding a digital still camera”, “an infrared sensor”, or “a distanceimage sensor”.

As an example, an imaging device that functions as the image sensordevice 102 includes a lens, an imaging element, and a signal processingcircuit.

The lens and the imaging element include, for example, an optical systemlens and an image sensor using a plurality of imaging elements, such asa complementary metal oxide semiconductor (CMOS) and a charge coupleddevice (CCD).

The signal processing circuit includes an automatic gain control (AGC)circuit and an analog to digital converter (ADC), and converts an analogsignal generated by the imaging element into a digital signal (imagedata). The signal processing circuit also performs various processesrelated to RAW development, for example. The signal processing circuitmay further perform various signal processing such as a white balancecorrection process, a color tone correction process, a gamma correctionprocess, a YCbCr conversion process, and an edge enhancement process.

Furthermore, the signal processing circuit may also perform processingrelated to the setting of the region for an image and may transfer theregion designation information to the IC chip 104. Furthermore, thesignal processing circuit may transfer various data such as exposureinformation and gain information to the IC chip 104.

The signal representing the image generated by the image sensor device102 is transferred to the IC chip 104. When the signal representing theimage transferred from the image sensor device 102 to the IC chip 104 isan analog signal, the IC chip 104 converts the analog signal into adigital signal by an ADC provided and processes the image data obtainedby the conversion. The following is a case where image data istransferred from the image sensor device 102 to the IC chip 104 as anexample.

The IC chip 104 is a chip formed with an integrated circuit (IC) inwhich a circuit related to the data transmission function using thefirst transmission method is integrated. The IC chip 104 is used toprocess image data transferred from the image sensor device 102 andtransmit the data corresponding to the generated image. The datacorresponding to an image is either image data transferred from theimage sensor device 102 (that is, data representing the entire image) orregion data concerning a region that is set in the image. The circuitrelated to the data transmission function according to the firsttransmission method is not limited to the implementation in the form ofone IC chip, and may be formed of a plurality of IC chips.

The IC chip 104 includes an image processing unit 106, a LINK/PHYcontroller 108, a control code generator 110, a PH generator 112, an EBDbuffer 114, an image data buffer 116, a combining unit 118, and atransmitting unit 120.

The image processing unit 106 can be formed as one circuit having afunction of performing processes related to the transmission methodaccording to the present embodiment. When performing the process relatedto the transmission method according to the present embodiment, theimage processing unit 106 controls, for each of lines in the image, totransmit region data corresponding to the region that is set in theimage, through the LINK/PHY controller 108, the control code generator110, the PH generator 112, the EBD buffer 114, the image data buffer116, the combining unit 118, and the transmitting unit 120, using thefirst transmission method. The image processing unit 106 can alsocontrol to transmit the image data transferred from the image sensordevice 102 (that is, the data representing the entire image) for each oflines.

The image processing unit 106 can be formed by a processor such as anMPU, for example.

The function of the image processing unit 106 will be described in theform of functional blocks. As illustrated in FIG. 10, the imageprocessing unit 106 includes a region cutout unit 122, an imageprocessing controller 124, and an encoder 126, for example.

The region cutout unit 122 has a function of performing a processrelated to the setting of a region for an image and thus sets a region(ROI) for the image represented by the image data transferred from theimage sensor device 102. The region cutout unit 122 performs a processrelated to region setting for an image, for example, according to theset operation mode. For example, the region cutout unit 122 performs theprocess related to the region setting for an image in a case where theoperation mode is an operation mode of transmitting region data.Furthermore, in a case where the operation mode is an operation mode oftransmitting data representing an entire image, the region cutout unit122 would not perform the process related to the region setting for theimage.

The region cutout unit 122 detects an object by performing an arbitraryobject detection process on an image and sets a region including thedetected object for each of the detected objects. Furthermore, theregion cutout unit 122 may set a region designated by an operation on anarbitrary operation device or the like. The region that is set by theregion cutout unit 122 can include a rectangular region such as theregions 1 and 2 in FIGS. 8 and 10 or a region of any shape other thanrectangular to be set to an image such as region 3 illustrated in FIGS.8 and 10.

In a case where the region is set, the region cutout unit 122 transfersregion designation information indicating the set region to the imageprocessing controller 124, for example. In a case where the region isnot set, the region cutout unit 122 would not transfer the regiondesignation information to the image processing controller 124.

The region cutout unit 122 also transfers image data transferred fromthe image sensor device 102 to the encoder 126.

The image processing controller 124 has a function of performing aprocess related to the transmission method according to the presentembodiment, and thus transfers the region information concerning theregion set for the image to the encoder 126 and the EBD buffer 114. Atthis time, the image processing controller 124 may set the additionalinformation other than the region information and may transfer theregion information and the other additional information to the EBDbuffer 114 as a series of additional information. Note that examples ofthe series of additional information including the region informationinclude information defined in Embedded Data described with reference toFIG. 6, for example.

An example of the process of setting additional information can be aprocess of generating additional information. Examples of the process ofgenerating additional information include one or two or more of theprocesses, namely, a process of generating information indicating theamount of data in the region, a process of generating informationindicating the size of the region, and a process of generatinginformation indicating the priority of the region.

Note that the process of setting additional information is not limitedto the process of generating the additional information. For example,the image processing controller 124 may set information acquired fromthe image sensor device 102, such as exposure information and gaininformation, as additional information. Furthermore, the imageprocessing controller 124 may set, as additional information, dataconcerning various regions, such as data indicating a physical regionlength, data indicating an output region length, data indicating animage format, and data indicating the total amount of data. Examples ofthe physical region length include the number of pixels of the imagesensor device 102. Examples of the output region length include thenumber of pixels of the image (length on the image) output from theimage sensor device 102.

The image processing controller 124 specifies a region included in eachof lines of the image on the basis of the region designation informationacquired from the region cutout unit 122 or region designationinformation (not illustrated) acquired from an external device, forexample. Subsequently, the image processing controller 124 sets theregion information based on the specified region.

In addition, the image processing controller 124 sets information (thatis, XY coordinates of the partial region) indicating the position,within the image, of the region included in each of lines (that is, thepartial region for each of lines). At this time, in a case where thereis no change from the previous position for transmission of the regiondata of the partial region in the horizontal direction (for example, theX coordinate at the left end), the image processing controller 124 doesnot need to set information (that is, the XY coordinates of the partialregion) indicating the position for the partial region for whichtransmission of the region data is to be performed. Subsequently, theimage processing controller 124 transfers information indicating theposition, within the image, of the region included in each of lines tothe encoder 126.

In addition, in a case where the region designation information is notacquired, the image processing controller 124 would not set the regioninformation.

Furthermore, the image processing controller 124 may generate frameinformation and transfer the generated frame information to the LINK/PHYcontroller 108, for example. Examples of the frame information includethe above-described Frame Start, Frame End, or the like, which areassigned to each of frames. Furthermore, frame information may includedata indicating data type such as YUV data, RGB data, or RAW data.

The encoder 126 encodes image data transferred from the image sensordevice 102 using a predetermined method such as a method correspondingto the joint photographic experts group (JPEG) standard, for example.

In a case where the region information is not acquired from the imageprocessing controller 124, the encoder 126 transfers the encoded imagedata to the image data buffer 116. Hereinafter, encoded image data, thatis, data indicating the entire encoded image will be referred to as“ordinary data” in some cases.

In addition, in a case where the region information is acquired from theimage processing controller 124, the encoder 126 transfers the encodedregion data corresponding to the region to the image data buffer 116according to the acquired region information. At this time, the encoder126 may associate each of pieces of region data with informationindicating the position, within the image, of the region (that is, thepartial region for each of lines) corresponding to the region data.Furthermore, at this time, in a case where the position of the targetpartial region in the horizontal direction (for example, the Xcoordinate of the left end) has not changed from the previous regiondata transmission position of the partial region in the horizontaldirection, the encoder 126 does not need to associate the region dataconcerning the target partial region with the information indicating theposition of the partial region. More specifically, the encoder 126 mayinsert information indicating the position of the corresponding partialregion at the top of the region data, as indicated by “XY” in theexamples illustrated in FIGS. 9 and 11.

The image processing unit 106 includes the region cutout unit 122, theimage processing controller 124, and the encoder 126, for example, andthereby performs the process related to the transmission methodaccording to the present embodiment. Note that the functional blocks ofthe image processing unit 106 illustrated in FIG. 12 are obtained bydividing the functions of the image processing unit 106 for convenience,and how to divide the functions in the image processing unit 106 is notlimited to the example illustrated in FIG. 12.

The LINK/PHY controller 108 transfers frame information to the controlcode generator 110 and the PH generator 112 for each of lines, forexample. Furthermore, at this time, the LINK/PHY controller 108 maycontrol the protocol and the transmission path related to the datatransmission and may transfer a result of the control to the controlcode generator 110 and the PH generator 112.

The control code generator 110 sets an error correction code for each oflines. The control code generator 110 generates an error correction codefor a line on the basis of data (for example, Frame Start or Frame End)for the line in the frame information. The control code generator 110transfers the generated error correction code to the combining unit 118,for example. Furthermore, the control code generator 110 may generate anerror correction code in cooperation with the PH generator 112.

The PH generator 112 uses the frame information to generate a packetheader for each of lines. At this time, in the case of transmittingregion data, the PH generator 112 sets information indicating that theregion information (for example, region data) is to be transmitted asthe header information type to the extension region of the packet headeras described above. Thereafter, the PH generator 112 sets informationindicating that the region data is to be transmitted using the payload,for at least a part of the extension region. Furthermore, regarding apacket in which the coordinates of the region are inserted to thepayload, the PH generator 112 sets, on at least a part the extensionregion, information indicating that the coordinates of the region are tobe transmitted using the payload.

The EBD buffer 114 is a buffer that temporarily holds additionalinformation transferred from the image processing unit 106 (the imageprocessing controller 124 in the example of FIG. 12). The EBD buffer 114outputs additional information to the combining unit 118 as “EmbeddedData” at a predetermined timing. The “Embedded Data” output from the EBDbuffer 114 may be transferred to the combining unit 118 via the imagedata buffer 116 described below.

The image data buffer 116 is a buffer that temporarily holds data(ordinary data or region data) transferred from the image processingunit 106 (encoder 126 in the example of FIG. 12). The image data buffer116 outputs the held data to the combining unit 118 for each of lines ata predetermined timing.

The combining unit 118 generates a packet to be transmitted on the basisof the data acquired individually from the control code generator 110,the PH generator 112, the EBD buffer 114, and the image data buffer 116,for example.

The transmitting unit 120 transmits the packets transferred from thecombining unit 118 via the data bus B1 (an example of a signaltransmission path; similar applies hereinafter) for each of lines.

For example, in a case where region 1, region 2, and region 3illustrated in FIG. 8 are set, the transmitting unit 120 transmits theregion information concerning each of regions as a part of additionalinformation (Embedded Data) using packet A2 while transmitting regiondata corresponding to each of regions using packet A1 for each of linesas illustrated in FIG. 9. Moreover, as another example, in a case whereregion 1, region 2, and region 3 illustrated in FIG. 10 are set, thetransmitting unit 120 transmits the region information concerning eachof regions as a part of additional information (Embedded Data) usingpacket A2 while transmitting region data corresponding to each ofregions using packet A1 for each of lines as illustrated in FIG. 11.

In addition, in a case where there is no region setting, that is, whereordinary data is output from the image data buffer 116, the transmittingunit 120 transmits, for each of lines, a packet having datacorresponding to each of lines being stored in the payload. Even in thiscase, the transmitting unit 120 can also transmit the additionalinformation as “Embedded Data”.

The above-described configuration is merely an example, and thefunctional configuration of the image sensor 100 is not limited to this.For example, the device or electronic device that executes each of theabove functions is not limited as long as each of the functions can berealized. As a more specific example, some of the functions of the ICchip 104 may be provided outside the IC chip 104.

Furthermore, additional functions may be added as long as the functionsof the image sensor 100 described above are not hindered.

As a specific example, an error correction code used for errorcorrection of the data may be calculated based on the data stored in thepayload of the packet (for example, the region data described above) andthe error correction code may be inserted as parity into the payload. Inthis manner, insertion of the parity into the payload enables thereceiving side to perform error correction calculation based on theparity so as to detect an error in the data stored in the payload andcorrect the detected error. Examples of the applicable error correctioncode include a Reed-Solomon code.

Hereinabove, an example of the functional configuration of the imagesensor 100 (transmission device) in the case of applying the firsttransmission method has been described with reference to FIG. 12.

Configuration Example of Processor 200 (Reception Device)

Next, an example of the functional configuration of the processor 200(reception device) in the case of applying the first transmission methodwill be described. For example, FIG. 13 is a block diagram illustratingan example of the functional configuration of the processor 200 in thecase of applying the first transmission method. As illustrated in FIG.13, the processor 200 includes a header separator 202, a headerinterpreter 204, a payload separator 206, an EBD interpreter 208, framememory 210, and an image processing unit 212.

The header separator 202 separates the header data corresponding to aheader portion and the payload data corresponding to the payload portionfrom the received data. The header separator 202 separates the headerdata from the received data according to a rule preliminarily defined bya standard or the like, for example. Furthermore, the header separator202 may separate the payload data from the received data according to arule preliminarily defined by the standard, for example, or may separatethe payload data from the received data based on the result of theprocessing performed by the header interpreter 204.

The header interpreter 204 interprets the content indicated by theheader data.

As a specific example, the header interpreter 204 recognizes the formatof the information that is set in a region other than top three bits ofthe extension region according to the header information type set in thetop three bits of the extension region of a packet header. Subsequently,the header interpreter 204 reads out various types of information set inthe extension region according to the recognition result of the format.This enables the header interpreter 204 to recognize transmission ofregion (ROI) information (for example, region data) or transmission ofcoordinates of the region using the payload on the basis of theinformation set in the extension region, for example. Subsequently, theheader interpreter 204 notifies the payload separator 206 of the settingrecognized according to a result of reading out various types ofinformation set in the extension region. Specifically, in a case wherethe header interpreter 204 has recognized transmission of the region(ROI) information (for example, region data) or transmission of thecoordinates of the region using the payload, the header interpreter 204notifies the payload separator 206 of the recognition result.

The example of the process performed on the header interpreter 204 isnot limited to the above example. For example, the header interpreter204 may specify the payload data position and transfer the specifiedposition to the header separator 202. The header interpreter 204 canalso distinguish between packet A2 that transmits “Embedded Data” andpacket A1 that transmits region data.

The payload separator 206 separates the additional information and theimage data (ordinary data or region data) from the payload data based onthe interpretation result from the header interpreter 204.

For example, in a case where the packet as a processing target is packetA2 or A3, the payload separator 206 may separate additional information(Embedded Data) from the packet.

Furthermore, as another example, in a case where the packet as aprocessing target is packet A1, the payload separator 206 separates theimage data from the payload data. For example, in a case where regiondata is stored in the payload, the payload separator 206 may separatethe region data from the payload data according to the packet headerinterpretation result. Furthermore, at this time, the payload separator206 may separate the coordinates of the region inserted into the toppart of the payload (that is, the XY coordinates of the partial region)according to the packet header interpretation result. In addition, in acase where ordinary data is stored in the payload, the payload separator206 may separate the ordinary data from the payload data according tothe packet header interpretation result.

The payload separator 206 transmits additional information out ofvarious types of data separated from the payload data, to the EBDinterpreter 208. Furthermore, the payload separator 206 causes the framememory 210 to hold image data (region data or ordinary data) out ofvarious type of data separated from the payload data. Furthermore, atthis time, the payload separator 206 may cause the frame memory 210 tohold the region data in association with the coordinates of the regioncorresponding to the region data (that is, the XY coordinates of thepartial region).

The EBD interpreter 208 interprets the content of additional information(Embedded Data) and then outputs the interpretation result of theadditional information to the image processing unit 212. Furthermore,the EBD interpreter 208 may cause the frame memory 210 to hold a resultof interpretation of the additional information. The format of theadditional information (Embedded Data) is as described above withreference to FIG. 6.

The image processing unit 212 includes an information extraction unit214, a region decoder 216, a region image generator 218, and an ordinaryimage decoder 220.

The information extraction unit 214 extracts various types ofinformation (particularly region information) included in the additionalinformation on the basis of a result of interpretation of the additionalinformation (Embedded Data). Subsequently, the information extractionunit 214 outputs various types of information extracted from theadditional information to the region decoder 216 and the region imagegenerator 218.

The region decoder 216 discriminates whether the image data held in theframe memory 210 is region data or ordinary data on the basis of theadditional information (Embedded Data) held in the frame memory 210. Ina case where the image data held in the frame memory 210 is region data,the region decoder 216 decodes the region data by a predetermined methodcorresponding to the encoding in the image sensor 100. At this time, theregion decoder 216 may change the content of the process based onvarious types of information extracted from the additional information(Embedded Data).

The region image generator 218 generates data representing an imagecorresponding to a region from region data for each of lines decoded bythe region decoder 216, based on various types of information (forexample, region information) extracted from the additional information(Embedded Data) and on the coordinates of the region associated with atleast a part of the region data.

For example, the region image generator 218 discriminates which region(ROI) includes each of pieces of region data corresponding to partialregion for each of lines, which is region data held in the frame memory210 corresponds, based on the region information included in theadditional information (Embedded Data). Furthermore, the region imagegenerator 218 recognizes the position, within the image, of each ofregions set in the image and the size of the region based on the regioninformation included in the additional information (Embedded Data).Subsequently, while adjusting the position of each of pieces of regiondata in the horizontal direction according to the coordinates of aregion associated with at least a part of the region data out of theregion data corresponding to each of the partial regions for each oflines in each of the regions, the region image generator 218 combines aseries of region data corresponding to the region. As described above,the region image generator 218 combines a series of region datacorresponding to each of regions, thereby generating a partial imagecorresponding to the region set in the image (hereinafter, also referredto as a “region image”). The region image corresponds to the “ROIimage”.

As described above, the region image generator 218 generates a regionimage for each of regions set in the image and outputs the generatedregion image to a predetermined output destination.

The ordinary image decoder 220 discriminates whether the image data heldin the frame memory 210 is region data or ordinary data on the basis ofthe additional information (Embedded Data) held in the frame memory 210.In a case where the image data held in the frame memory 210 is ordinarydata, the ordinary image decoder 220 decodes the ordinary data by apredetermined method corresponding to the encoding in the image sensor100 to generate an ordinary image and then outputs the generatedordinary image to a predetermined output destination.

The above-described configuration is merely an example, and thefunctional configuration of the processor 200 is not limited to this.For example, the device or electronic device that executes each of theabove functions is not limited as long as each of the functions can berealized. As a more specific example, some of the functions of theprocessor 200 may be provided outside the processor 200.

Furthermore, additional functions may be added as long as the functionsof the processor 200 described above are not hindered.

For example, in a case where an error correction code has been added toa part of the data of the packet, various processes may be executedbased on the error correction code. As a specific example, an errorcorrection code calculated from the data stored in the payload of thepacket may be inserted as parity into the packet load. In this case, itis possible to detect an error in the data stored in the payload (forexample, the above-described region data) and correct the detected errorbased on the error correction calculation based on the parity.

(Evaluation)

As described above, in the first transmission method, the regioninformation of each of regions set in the image is transmitted in packetA2 as a part of additional information (Embedded Data), and the regiondata corresponding to each of regions is transmitted for each of linesin packet A1. Furthermore, in a case of transmitting the data of theregion set in the image, information indicating that the regioninformation (for example, region data) is to be transmitted is set inthe extension region of the header of each of packets. For a packet inwhich the coordinates of the region are inserted in the payload,information indicating that the coordinates of the region are to betransmitted using the payload is set in at least a part of the extensionregion. With the above configuration, the receiving side combines regiondata stored in the payload of each of packets A1 based on the additionalinformation (Embedded Data) and the coordinates of the region insertedin at least part of the payload of packet A1, making it possible toeasily restore the region image of the region set in the image.

<3.2. Second Transmission Method>

Next, a second transmission method will be described. Similarly to thefirst transmission method, the image sensor 100 in the secondtransmission method stores region data concerning the region set in theimage in the payload of the packet and transmits the data for each oflines. Therefore, in the following, the second transmission method willbe described focusing on the part different from the first transmissionmethod, and the detailed description of the parts that are substantiallysimilar to the first transmission method will be omitted.

(Data Format)

First, the format of data transmitted by the second transmission methodwill be described. For example, FIG. 14 is a diagram illustrating theformat of data transmitted by the second transmission method. Note that,in FIG. 14, the similar reference numerals as those in FIG. 5 denote thesimilar objects as those in the example illustrated in FIG. 5.

In the example illustrated in FIG. 14, at least a part of the series ofpackets A2 is used for transmitting Embedded Data. For example, EmbeddedData may be transmitted in a state of being stored in the payload ofpacket A2. Furthermore, as another example, the embedded data may betransmitted in a state of being stored in a region other than thepayload of packet A2. Note that the Embedded Data in the secondtransmission method corresponds to additional information (in otherwords, information embedded by the image sensor 100) additionallytransmitted by the image sensor 100, similarly to the first transmissionmethod.

For example, FIG. 15 is a diagram illustrating an example of informationincluded in Embedded Data in the second transmission method. Inaddition, as illustrated in FIG. 15, each of items, the “ROI ID”, “upperleft coordinate”, “height”, “width”, “AD word length (AD bit)”,“exposure”, “gain”, and “sensing information” illustrates informationsimilar to the example illustrated in FIG. 6.

In addition, the second transmission method differs from the firsttransmission method in that region data concerning a partial regioncorresponding to each of a plurality of regions in a line in which theplurality of regions separated from each other in the horizontaldirection is set is transmitted in one packet A1. Therefore, in thiscase, the payload of packet A1 stores the region data of the partialregions corresponding to each of the plurality of regions. Furthermore,at this time, arbitrary data corresponding to an interval between thepartial regions in the horizontal direction is inserted between thepieces of region data corresponding to the partial regions of each ofthe plurality of regions separated from each other in the horizontaldirection. Note that, hereinafter, the arbitrary data is also referredto as “blank data”. For example, “BLANK” illustrated in FIG. 14indicates blank data.

The data inserted as blank data is not particularly limited. Forexample, image data between the partial regions may be inserted as blankdata. Furthermore, data other than the image data (for example, datahave a setting of a value that cannot be image data) may be inserted asblank data.

Furthermore, in the second transmission method, XY coordinates of thepartial region can be inserted at the top of the payload of packet A1 asindicated by “XY”. Note that in a case where the payload stores regiondata corresponding to a partial region of each of a plurality ofregions, the X and Y coordinates indicating the position of the left endof the top partial region (partial region of the left end) are insertedas the above-described XY coordinates of the partial region. Inaddition, in a case where there is no change in the X coordinates of thecorresponding partial regions between consecutively transmitted packetsA1 and the Y coordinate is incremented by just 1, the XY coordinates ofthe partial regions may be omitted.

Next, with reference to FIG. 16, an example of a configuration of thepacket header of packet A1 used for transmitting the region data of theregion (ROI) set in the image will be described, focusing particularlyon the configuration of the extension region. FIG. 16 is a diagramillustrating an example of a configuration of the packet header in thesecond transmission method.

As illustrated in FIG. 16, in the case of transmitting the region dataof the region (ROI) set in the image in the second transmission method,information indicating that the region information is to be transmitted(that is, the information corresponding to the type assuming thetransmission of the region information) is set as a header informationtype in the packet header of packet A1 used for transmitting the regiondata. Furthermore, information indicating that the region data (that is,region data concerning the partial region) is to be transmitted usingthe payload is set to at least a part of the extension region. Inaddition, in the case of transmitting the coordinates of the region(that is, the XY coordinates of the partial region) using the payload,information indicating that the coordinates of the region are to betransmitted is set for at least a part of the extension region. In thecase of transmitting the region data concerning the region (ROI) set inthe image, the payload length of packet A1 can change according to thewidth of the region in the horizontal direction. Therefore, informationindicating the payload length may be set in a part of the extensionregion, similarly to the example described with reference to FIG. 4.

Example of Data

Next, with reference to FIGS. 17 and 18, a specific example of datatransmitted by the first transmission method will be described as anexample in a case where the second transmission method is applied.

For example, FIG. 17 is a diagram illustrating an example of datatransmitted by the second transmission method, illustrating one-framedata in a case where the region data concerning each of the regionsillustrated in FIG. 8 is transmitted by the second transmission method.Note that the configurations of packets A2 and A3 in FIG. 17 aresubstantially similar to the example described with reference to FIG. 5,and hence the following description will focus on the configuration ofpacket A1. Furthermore, each of “1”, “2”, and “3” illustrated in FIG. 17corresponds to the data of region 1, the data of region 2, and the dataof region 3 stored in the payload of packet A1.

As described above, the second transmission method differs from thefirst transmission method in that region data concerning a partialregion corresponding to each of a plurality of regions in a line inwhich the plurality of regions separated from each other in thehorizontal direction is set is transmitted in one packet A1. Therefore,like the lines corresponding to the third to fifth lines of region 1,there is a difference from the example described with reference to FIG.9 in the data stored in the payload of packet A1 transmitted for theline set so that the partial region of region 1 and the partial regionof region 2 are horizontally separated from each other. That is, thepayload of packet A1 corresponding to the line stores region data of thepartial regions of individual regions of region 1 and region 2.Furthermore, in the payload, blank data corresponding to an interval inthe horizontal direction between the partial regions is inserted betweenpieces of region data of the partial regions of region 1 and region 2.For example, the portion illustrated as “Dummy” in FIG. 17 correspondsto blank data.

Furthermore, with the above configuration, regarding packet A1transmitted for each of lines corresponding to the first to fifth linesof the first region, the X coordinate of the left end of the top regiondata out of the region data stored in the payload indicates a samevalue, with the Y coordinate incremented by 1 from the immediatelypreceding line. Therefore, regarding packet A1 transmitted for each oflines corresponding to the second to fifth lines of the first region,the insertion of the XY coordinates of the partial region indicated by“XY” to the top of the payload is omitted.

Furthermore, FIG. 18 is a diagram illustrating an example of datatransmitted by the second transmission method, illustrating one-framedata in a case where the region data concerning each of the regionsillustrated in FIG. 10 is transmitted by the second transmission method.Note that the configurations of packets A2 and A3 in FIG. 18 as well aresubstantially similar to the example described with reference to FIG. 5,and hence the description will focus on the configuration of packet A1.Furthermore, each of “1”, “2”, and “3” illustrated in FIG. 18corresponds to the data of region 1, the data of region 2, and the dataof region 3 stored in the payload of packet A1.

In the example illustrated in FIG. 10, there is no line in which aplurality of regions separated from each other in the horizontaldirection is set. Therefore, the data illustrated in FIG. 18 issubstantially similar to the data when the first transmission methodillustrated in FIG. 11 is applied.

Example of Configuration of Image Sensor 100 (Transmission Device)

Next, an example of a functional configuration of the image sensor 100(transmission device) in the case of applying the second transmissionmethod will be described. For example, FIG. 19 is a block diagramillustrating an example of a functional configuration of the imagesensor 100 in the case of applying the second transmission method. Asillustrated in FIG. 19, the image sensor 100 includes an image sensordevice 102 and an IC chip 130, for example. The image sensor 100 in FIG.19 operates on electric power supplied from an internal power supply(not illustrated) such as a battery included in the communication system1000, or electric power supplied from an external power supply of thecommunication system 1000.

The image sensor device 102 illustrated in FIG. 19 is substantiallysimilar to the image sensor device 102 illustrated in FIG. 12.

The IC chip 130 is a chip formed with an IC in which a circuit relatedto the data transmission function using the second transmission methodis integrated and is used to process image data transferred from theimage sensor device 102 and transmit the data corresponding to thegenerated image. The circuit related to the data transmission functionaccording to the second transmission method is not limited to theimplementation in the form of one IC chip, and may be formed of aplurality of IC chips.

The IC chip 130 includes an image processing unit 132, a LINK/PHYcontroller 108, a control code generator 110, a PH generator 134, an EBDbuffer 138, an image data buffer 116, a combining unit 136, and atransmitting unit 120.

The LINK/PHY controller 108, the control code generator 110, the imagedata buffer 116, and the transmitting unit 120 illustrated in FIG. 19are respectively substantially similar to the LINK/PHY controller 108,the control code generator 110, the image data buffer 116, and thetransmitting unit 120 illustrated in FIG. 12.

The image processing unit 132 can be formed as one circuit having afunction of performing processes related to the transmission methodaccording to the present embodiment. When performing the process relatedto the transmission method according to the present embodiment, theimage processing unit 132 controls, for each of lines in the image, totransmit region data corresponding to the region set in the image,through the LINK/PHY controller 108, the control code generator 110, thePH generator 134, the EBD buffer 138, the image data buffer 116, thecombining unit 136, and the transmitting unit 120, using the secondtransmission method. The image processing unit 132 can also control totransmit the image data transferred from the image sensor device 102(that is, the data representing the entire image) for each of lines.

The image processing unit 132 can be formed by a processor such as anMPU, for example.

The function of the image processing unit 106 will be described in theform of functional blocks. As illustrated in FIG. 10, the imageprocessing unit 106 includes a region cutout unit 122, an imageprocessing controller 124, and an encoder 140, for example.

The region cutout unit 122 and the image processing controller 124illustrated in FIG. 19 are respectively substantially similar to theregion cutout unit 122 and the image processing controller 124illustrated in FIG. 12.

Similar to the encoder 126 illustrated in FIG. 12, the encoder 140encodes image data transferred from the image sensor device 102 by apredetermined method.

In a case where the region information is not acquired from the imageprocessing controller 124, the encoder 140 transfers the encoded imagedata to the image data buffer 116, similarly to the encoder 126illustrated in FIG. 12.

In addition, in a case where the region information is acquired from theimage processing controller 124, the encoder 140 transfers the encodedregion data corresponding to the region to the image data buffer 116according to the acquired region information. At this time, the encoder140 may associate each of pieces of region data with informationindicating the position, within the image, of the region (that is, thepartial region for each of lines) corresponding to the region data.Furthermore, at this time, in a case where the position of the targetpartial region in the horizontal direction (for example, the Xcoordinate of the left end) has not changed from the previous regiondata transmission position of the partial region in the horizontaldirection, the encoder 140 does not need to associate the region dataconcerning the target partial region with the information indicating theposition of the partial region. More specifically, the encoder 140 mayinsert information indicating the position of the corresponding partialregion at the top of the region data, as indicated by “XY” in theexamples illustrated in FIGS. 17 and 18.

Furthermore, the encoder 140 encodes the region data concerning thepartial regions corresponding to each of the plurality of regions as aseries of data for a line in which a plurality of regions separated fromeach other in the horizontal direction is set. Subsequently, the encoder140 transfers the data obtained by encoding the series of region dataincluded in the line to the image data buffer 116. In this case, theencoder 140 may associate the data obtained by encoding the series ofregion data included in the line with information indicating theposition in the image of the partial region corresponding to the topregion data out of the series of region data.

The image processing unit 132 includes the region cutout unit 122, theimage processing controller 124, and the encoder 140, for example, andthereby performs the process related to the transmission methodaccording to the present embodiment. Note that the functional blocks ofthe image processing unit 132 illustrated in FIG. 19 are obtained bydividing the functions of the image processing unit 132 for convenience,and how to divide the functions in the image processing unit 132 is notlimited to the example illustrated in FIG. 19.

The PH generator 134 uses the frame information to generate a packetheader for each of lines. At this time, in the case of transmittingregion data, the PH generator 134 sets information indicating that theregion information (for example, region data) is to be transmitted asthe header information type to the extension region of the packet headeras described above. Thereafter, the PH generator 134 sets informationindicating that the region data is to be transmitted using the payload,for at least a part of the extension region. Furthermore, regarding apacket in which the coordinates of the region are inserted to thepayload, the PH generator 134 sets, on at least a part the extensionregion, information indicating that the coordinates of the region are tobe transmitted using the payload.

The EBD buffer 138 is a buffer that temporarily holds additionalinformation transferred from the image processing unit 132 (the imageprocessing controller 124 in the example of FIG. 19). The EBD buffer 138outputs additional information to the combining unit 136 as “EmbeddedData” at a predetermined timing. The “Embedded Data” output from the EBDbuffer 138 may be transferred to the combining unit 136 via the imagedata buffer 116.

The combining unit 136 generates a packet to be transmitted, based onthe data acquired individually from the control code generator 110, thePH generator 134, the EBD buffer 138, and the image data buffer 116, forexample.

The transmitting unit 120 transmits the packet transferred from thecombining unit 136 for each of lines via the data bus B1.

For example, in a case where region 1, region 2, and region 3illustrated in FIG. 8 are set, the transmitting unit 120 transmits theregion information concerning each of regions as a part of additionalinformation (Embedded Data) using packet A2 while transmitting regiondata corresponding to each of regions using packet A1 for each of linesas illustrated in FIG. 17. Moreover, as another example, in a case whereregion 1, region 2, and region 3 illustrated in FIG. 10 are set, thetransmitting unit 120 transmits the region information concerning eachof regions as a part of additional information (Embedded Data) usingpacket A2 while transmitting region data corresponding to each ofregions using packet A1 for each of lines as illustrated in FIG. 18.

In addition, in a case where there is no region setting, that is, whereordinary data is output from the image data buffer 116, the transmittingunit 120 transmits, for each of lines, a packet having datacorresponding to each of lines being stored in the payload. Even in thiscase, the transmitting unit 120 can also transmit the additionalinformation as “Embedded Data”.

The above-described configuration is merely an example, and thefunctional configuration of the image sensor 100 is not limited to this.For example, the device or electronic device that executes each of theabove functions is not limited as long as each of the functions can berealized. As a more specific example, some of the functions of the ICchip 130 may be provided outside the IC chip 130.

Furthermore, additional functions may be added as long as the functionsof the image sensor 100 described above are not hindered, similarly tothe above-described first transmission method.

Hereinabove, an example of the functional configuration of the imagesensor 100 (transmission device) in the case of applying the secondtransmission method has been described with reference to FIG. 19.

Configuration Example of Processor 200 (Reception Device)

Next, an example of the functional configuration of the processor 200(reception device) in the case of applying the second transmissionmethod will be described. For example, FIG. 20 is a block diagramillustrating an example of the functional configuration of the processor200 in the case of applying the second transmission method. Asillustrated in FIG. 20, the processor 200 includes, for example, aheader separator 232, a header interpreter 234, a payload separator 236,an EBD interpreter 238, frame memory 210, and an image processing unit240.

The header separator 232 separates the header data corresponding to aheader portion and the payload data corresponding to the payload portionfrom the received data. The header separator 232 separates the headerdata from the received data according to a rule preliminarily defined bya standard or the like, for example. Furthermore, the header separator232 may separate the payload data from the received data according to arule preliminarily defined by the standard, for example, or may separatethe payload data from the received data based on the result of theprocessing performed by the header interpreter 234.

The header interpreter 234 interprets the content indicated by theheader data.

As a specific example, the header interpreter 234 recognizes the formatof the information that is set in a region other than top three bits ofthe extension region according to the header information type set in thetop three bits of the extension region of a packet header. Subsequently,the header interpreter 234 reads out various types of information set inthe extension region according to the recognition result of the format.This enables the header interpreter 234 to recognize transmission ofregion (ROI) information (for example, region data) or transmission ofcoordinates of the region using the payload, based on the informationset in the extension region, for example. Subsequently, the headerinterpreter 234 notifies the payload separator 236 of the settingrecognized according to a result of reading out various types ofinformation set in the extension region. Specifically, in a case wherethe header interpreter 234 has recognized transmission of the region(ROI) information (for example, region data) or transmission of thecoordinates of the region using the payload, the header interpreter 234notifies the payload separator 236 of the recognition result.

The example of the process performed on the header interpreter 234 isnot limited to the above example. For example, the header interpreter234 may specify the position of the payload data and may transfer thespecified position to the header separator 232. The header interpreter234 can also distinguish between packet A2 that transmits “EmbeddedData” and packet A1 that transmits region data.

The payload separator 236 separates the additional information and theimage data (ordinary data or region data) from the payload data based onthe interpretation result from the header interpreter 234.

For example, in a case where the packet as a processing target is packetA2 or A3, the payload separator 236 may separate additional information(Embedded Data) from the packet.

Furthermore, as another example, in a case where the packet as aprocessing target is packet A1, the payload separator 236 separates theimage data from the payload data. For example, in a case where regiondata is stored in the payload, the payload separator 236 may separatethe region data from the payload data according to the packet headerinterpretation result. Furthermore, at this time, the payload separator236 may separate the coordinates of the region inserted into the toppart of the payload (that is, the XY coordinates of the partial region)according to the packet header interpretation result. In addition, in acase where ordinary data is stored in the payload, the payload separator236 may separate the ordinary data from the payload data according tothe packet header interpretation result.

The payload separator 236 transmits additional information out ofvarious types of data separated from the payload data, to the EBDinterpreter 238. Furthermore, the payload separator 236 causes the framememory 210 to hold image data (region data or ordinary data) out ofvarious type of data separated from the payload data. Furthermore, atthis time, the payload separator 236 may cause the frame memory 210 tohold the region data in association with the coordinates of the regioncorresponding to the region data (that is, the XY coordinates of thepartial region).

The EBD interpreter 238 interprets the content of additional information(Embedded Data) and then outputs the interpretation result of theadditional information to the image processing unit 240. Furthermore,the EBD interpreter 238 may cause the frame memory 210 to hold a resultof interpretation of the additional information. The format of theadditional information (Embedded Data) is as described above withreference to FIG. 15.

The image processing unit 240 includes an information extraction unit242, a region decoder 244, a region image generator 246, and an ordinaryimage decoder 220.

The information extraction unit 242 extracts various types ofinformation (particularly region information) included in the additionalinformation on the basis of a result of interpretation of the additionalinformation (Embedded Data). Subsequently, the information extractionunit 242 outputs various types of information extracted from theadditional information to the region decoder 244 and the region imagegenerator 246.

The region decoder 244 discriminates whether the image data held in theframe memory 210 is region data or ordinary data on the basis of theadditional information (Embedded Data) held in the frame memory 210. Ina case where the image data held in the frame memory 210 is region data,the region decoder 244 decodes the region data by a predetermined methodcorresponding to the encoding in the image sensor 100. At this time, theregion decoder 244 may change the content of the process based onvarious types of information extracted from the additional information(Embedded Data).

The region image generator 246 generates data representing an imagecorresponding to a region from region data for each of lines decoded bythe region decoder 244, based on various types of information (forexample, region information) extracted from the additional information(Embedded Data) and on the coordinates of the region associated with atleast a part of the region data.

For example, the region image generator 246 discriminates which region(ROI) includes each of pieces of region data corresponding to partialregion for each of lines, which is region data held in the frame memory210, based on the region information included in the additionalinformation (Embedded Data). Furthermore, the region image generator 246recognizes the position, within the image, of each of regions set in theimage and the size of the region, based on the region informationincluded in the additional information (Embedded Data). Subsequently,while adjusting the position of each of pieces of region data in thehorizontal direction according to the coordinates of a region associatedwith at least a part of the region data out of the region datacorresponding to each of the partial regions for each of lines in eachof the regions, the region image generator 246 combines a series ofregion data corresponding to the region. As described above, the regionimage generator 246 combines a series of region data corresponding toeach of regions, thereby generating a region image corresponding to theregion set in the image.

In addition, regarding a line in which a plurality of regions separatedfrom each other in the horizontal direction is set, the region imagegenerator 246 recognizes the position of another region other than thetop region as a relative position with respect to the top region, basedon the blank data inserted between the pieces of region datacorresponding to each of regions. Subsequently, while adjusting positionof each of pieces of the region data of the region in the horizontaldirection based on the recognition result for the position of each ofregions, the region image generator 246 combines the series of regiondata corresponding to the region, thereby generating a region imagecorresponding to the region.

As described above, the region image generator 246 generates a regionimage for each of regions set in the image and outputs the generatedregion image to a predetermined output destination.

The ordinary image decoder 220 illustrated in FIG. 20 is substantiallysimilar to the ordinary image decoder 220 illustrated in FIG. 13.

The above-described configuration is merely an example, and thefunctional configuration of the processor 200 is not limited to this.For example, the device or electronic device that executes each of theabove functions is not limited as long as each of the functions can berealized. As a more specific example, some of the functions of theprocessor 200 may be provided outside the processor 200.

Furthermore, additional functions may be added as long as the functionsof the processor 200 described above are not hindered, similarly to theabove-described first transmission method.

(Evaluation)

As described above, in the second transmission method, the regioninformation of each of regions set in the image is transmitted in packetA2 as a part of additional information (Embedded Data), and the regiondata corresponding to each of regions is transmitted for each of linesin packet A1. Furthermore, in a case of transmitting the data of theregion set in the image, information indicating that the regioninformation (for example, region data) is to be transmitted is set inthe extension region of the header of each of packets. For a packet inwhich the coordinates of the region are inserted in the payload,information indicating that the coordinates of the region are to betransmitted using the payload is set in at least a part of the extensionregion. In addition, regarding a line in which a plurality of regionsseparated from each other in the horizontal direction is set, theposition of another region other than the top region is recognized as arelative position with respect to the top region, based on the blankdata inserted between the pieces of region data corresponding to each ofregions. With the above configuration, the receiving side combinesregion data stored in the payload of each of packets A1 based on theadditional information (Embedded Data), coordinates of the regioninserted in at least part of the payload of packet A1, and a detectionresult of the blank data, making it possible to easily restore theregion image of the region set in the image. In the second transmissionmethod, the region data included in one line is transmitted in onepacket A1. This makes it possible to easily calculate the maximum lengthof the data for one line.

<3.3. Third Transmission Method>

Next, a third transmission method will be described. Similarly to theabove-described transmission methods, the image sensor 100 in the thirdtransmission method stores region data concerning the region set in theimage in the payload of the packet and transmits the data for each oflines. Therefore, in the following, the third transmission method willbe described focusing on the part different from the first transmissionmethod, and the detailed description of the parts that are substantiallysimilar to the first transmission method will be omitted.

(Data Format)

First, the format of data transmitted by the third transmission methodwill be described. For example, FIG. 21 is a diagram illustrating theformat of data transmitted by the third transmission method. Note that,in FIG. 21, the similar reference numerals as those in FIG. 5 denote thesimilar objects as those in the example illustrated in FIG. 5.

In the example illustrated in FIG. 21, at least a part of the series ofpackets A2 is used for transmitting Embedded Data. For example, EmbeddedData may be transmitted in a state of being stored in the payload ofpacket A2. Furthermore, as another example, the embedded data may betransmitted in a state of being stored in a region other than thepayload of packet A2. Note that the Embedded Data in the thirdtransmission method corresponds to additional information (in otherwords, information embedded by the image sensor 100) additionallytransmitted by the image sensor 100, similarly to the first transmissionmethod.

For example, FIG. 22 is a diagram illustrating an example of informationincluded in Embedded Data in the third transmission method. In addition,as illustrated in FIG. 22, each of items, the “ROI ID”, “upper leftcoordinate”, “height”, “width”, “AD word length (AD bit)”, “exposure”,“gain”, and “sensing information” illustrates information similar to theexample illustrated in FIG. 6.

The third transmission method differs from the above-described firsttransmission method in that the information indicating the position ofthe partial region corresponding to the region data stored in thepayload (that is, the XY coordinates of the partial region) is notinserted in the payload. Furthermore, even when one line includes aplurality of regions, the region data of the partial regionscorresponding to each of the plurality of regions is defined as a seriesof data, and the series of data is transmitted in one packet A1.

Next, with reference to FIG. 23, an example of a configuration of thepacket header of packet A1 used for transmitting the region data of theregion (ROI) set in the image will be described, focusing particularlyon the configuration of the extension region. FIG. 23 is a diagramillustrating an example of a configuration of the packet header in thethird transmission method.

As illustrated in FIG. 23, in the case of transmitting the region dataof the region (ROI) set in the image in the third transmission method,information indicating that the region information is to be transmitted(that is, the information corresponding to the type assuming thetransmission of the region information) is set as a header informationtype in the packet header of packet A1 used for transmitting the regiondata. Furthermore, information indicating that the region data (that is,region data concerning the partial region) is to be transmitted usingthe payload is set to at least a part of the extension region. In thecase of transmitting the region data concerning the region (ROI) set inthe image, the payload length of packet A1 can change according to thewidth of the region in the horizontal direction. Therefore, informationindicating the payload length may be set in a part of the extensionregion, similarly to the example described with reference to FIG. 4.

Example of Data

Next, with reference to FIGS. 24 and 25, a specific example of datatransmitted by the first transmission method will be described as anexample in a case where the second transmission method is applied.

For example, FIG. 24 is a diagram illustrating an example of datatransmitted by the third transmission method, illustrating one-framedata in a case where the region data concerning each of the regionsillustrated in FIG. 8 is transmitted by the third transmission method.Note that the configurations of packets A2 and A3 in FIG. 24 aresubstantially similar to the example described with reference to FIG. 5,and hence the following description will focus on the configuration ofpacket A1. Furthermore, each of “1”, “2”, and “3” illustrated in FIG. 24corresponds to the data of region 1, the data of region 2, and the dataof region 3 stored in the payload of packet A1.

As described above, unlike the above-described first transmissionmethod, information indicating the position of the partial regioncorresponding to the region data stored in the payload (that is, XYcoordinates of partial region) will not be inserted to the payload ofpacket A1 in the third transmission method. Furthermore, even when oneline includes a plurality of regions, the region data of the partialregions corresponding to each of the plurality of regions is defined asa series of data, and the series of data is stored in the single payloadof packet A1. Having such characteristics, in the example illustrated inFIG. 24, the payload of packet A1 stores only the region datacorresponding to the partial region of the region included in each oflines.

Furthermore, FIG. 25 is a diagram illustrating an example of datatransmitted by the third transmission method, illustrating one-framedata in a case where the region data concerning each of the regionsillustrated in FIG. 10 is transmitted by the third transmission method.Note that the configurations of packets A2 and A3 in FIG. 25 as well aresubstantially similar to the example described with reference to FIG. 5,and hence the description will focus on the configuration of packet A1.Furthermore, each of “1”, “2”, and “3” illustrated in FIG. 25corresponds to the data of region 1, the data of region 2, and the dataof region 3 stored in the payload of packet A1.

In the example illustrated in FIG. 25, it is also known that the payloadof packet A1 stores only the region data corresponding to the partialregion of the region included in each of lines.

Example of Configuration of Image Sensor 100 (Transmission Device)

Next, an example of a functional configuration of the image sensor 100(transmission device) in the case of applying the third transmissionmethod will be described. For example, FIG. 26 is a block diagramillustrating an example of a functional configuration of the imagesensor 100 in the case of applying the third transmission method. Asillustrated in FIG. 26, the image sensor 100 includes an image sensordevice 102 and an IC chip 300, for example. The image sensor 100 in FIG.26 operates on electric power supplied from an internal power supply(not illustrated) such as a battery included in the communication system1000, or electric power supplied from an external power supply of thecommunication system 1000.

The image sensor device 102 illustrated in FIG. 26 is substantiallysimilar to the image sensor device 102 illustrated in FIG. 12.

The IC chip 300 is a chip formed with an IC in which a circuit relatedto the data transmission function using the third transmission method isintegrated and is used to process image data transferred from the imagesensor device 102 and transmit the data corresponding to the generatedimage. The circuit related to the data transmission function accordingto the third transmission method is not limited to the implementation inthe form of one IC chip, and may be formed of a plurality of IC chips.

The IC chip 300 includes an image processing unit 302, a LINK/PHYcontroller 304, a control code generator 306, a PH generator 308, an EBDbuffer 310, an image data buffer 312, a combining unit 314, and atransmitting unit 328.

The image processing unit 302 can be formed as one circuit having afunction of performing processes related to the transmission methodaccording to the present embodiment. When performing the process relatedto the transmission method according to the present embodiment, theimage processing unit 302 controls, for each of lines in the image, totransmit region data corresponding to the region set in the image,through the LINK/PHY controller 304, the control code generator 306, thePH generator 308, the EBD buffer 310, the image data buffer 312, thecombining unit 314, and the transmitting unit 328, using the thirdtransmission method. The image processing unit 302 can also control totransmit the image data transferred from the image sensor device 102(that is, the data representing the entire image) for each of lines.

The image processing unit 302 can be formed by a processor such as anMPU, for example.

The function of the image processing unit 302 will be described in theform of functional blocks. As illustrated in FIG. 26, the imageprocessing unit 302 includes a region cutout unit 316, a region analyzer318, an overlap detector 320, a priority setting unit 322, an imageprocessing controller 326, and an encoder 324, for example.

The region cutout unit 316 has a function of performing a processrelated to the setting of a region for an image, and thus sets a region(ROI) for an image indicated by the image data transferred from theimage sensor device 102. The region cutout unit 316 performs a processrelated to region setting for an image, for example, according to theset operation mode. For example, the region cutout unit 316 performs theprocess related to the region setting for an image in a case where theoperation mode is an operation mode of transmitting region data.Furthermore, in a case where the operation mode is an operation mode oftransmitting data representing an entire image, the region cutout unit316 would not perform the process related to the region setting for theimage.

The region cutout unit 316 specifies one or more objects (subjects) asan imaging target included in the image represented by the image datatransferred from the image sensor device 102 (hereinafter, also referredto as “captured image”) and then sets a region (ROI) for each of thespecified objects. The region that is set by the region cutout unit 316can include a rectangular region such as the regions 1 and 2 in FIGS. 8and 10 or a region of any shape other than rectangular to be set to animage such as region 3 illustrated in FIGS. 8 and 10.

The region cutout unit 316 cuts out an image of each of regions (thatis, a region image) from the image sensor device 102. The region cutoutunit 316 further assigns a region number (ROI ID) as an identifier toeach of the set regions. For example, in a case where two regions areset in the image sensor device 102, the region cutout unit 316 assignsregion number 1 to one region and assigns region number 2 to the otherregion.

In a case where the region is set, the region cutout unit 316 transfersregion designation information indicating the set region to the imageprocessing controller 326 or the region analyzer 318, for example. In acase where the region is not set, the region cutout unit 316 would nottransfer the region designation information to the image processingcontroller 326 or the region analyzer 318.

The region cutout unit 316 also transfers image data transferred fromthe image sensor device 102 to the encoder 324.

The region analyzer 318 derives the position of the region in an imagefor each of regions (ROI) set in the image. The position of the regionis defined by upper left corner coordinates (Xa, Ya) of the region, alength (that is, a width) of the region in the horizontal direction (xdirection), and a length (that is, a height) of the region in thevertical direction (y direction). An example of the length of the regionin the horizontal direction is a physical region length XLa in the xdirection of the region. An example of the length of the region in theY-axis direction is a physical region length YLa of the region in theY-axis direction. The physical region length refers to the physicallength (data length) of the region. The position information may includecoordinates of a position different from the upper left end of theregion. The region analyzer 318 stores the derived position informationin the storage unit, for example. The region analyzer 318 stores theinformation in the storage unit in association with an identifier(region number) assigned to the region, for example.

The region analyzer 318 may further derive, as position information, anoutput region length XLc in the horizontal direction (x direction) ofthe region and an output region length YLc in the vertical direction (ydirection) of the region, for each of regions, for example. An exampleof the output region length is a physical length (data length) of aregion (ROI) after the resolution is changed by thinning processing orpixel addition. In addition to position information, the region analyzer318 may derive “AD word length”, “exposure”, “gain”, “sensinginformation”, “image format”, or the like for each of regions, so as tobe stored in the storage unit. The image format refers to the format ofthe region image. The region analyzer 318 may derive, for example, thenumber of regions (ROIs) included in the captured image (ROI number) andstore the derived number in the storage unit.

In a case where a plurality of objects as an imaging target is specifiedin a captured image, the overlap detector 320 detects an overlappingregion where two or more regions overlap (region of overlap (ROO)) onthe basis of the position information of the plurality of regions in thecaptured image. That is, the overlap detector 320 derives the positioninformation of the overlapping region ROO in the captured image for eachof the overlapping regions ROO. The overlap detector 320 stores, forexample, the derived position information of the overlapping region ROOin the storage unit. The overlap detector 320 stores the derivedposition information of the overlapping region ROO in the storage unitin association with the overlapping region ROO, for example. Theoverlapping region ROO is, for example, a region having the same size asor smaller than a smallest region in two or more regions (ROI)overlapping each other or a region smaller than this (for example, arectangular region). The position information of the overlapping regionROO is defined by upper left corner coordinates (Xb, Yb) of theoverlapping region ROO, a length (that is, a width) of the overlappingregion ROO in the horizontal direction (x direction), and a length (thatis, a height) of the overlapping region ROO in the vertical direction (ydirection). An example of the length of the overlapping region ROO inthe horizontal direction is a physical region length XLb. An example ofthe length of the overlapping region ROO in the vertical direction is aphysical region length YLb. The position information of the overlappingregion ROO may include coordinates of a position different from theupper left end of the region (ROI).

The priority setting unit 322 assigns a priority to each of regions(ROI) in a captured image. The priority setting unit 322 stores theassigned priority in the storage unit, for example. The priority settingunit 322 stores the assigned priority in the storage unit in associationwith the region, for example. The priority setting unit 322 may assign apriority to each of regions in addition to the region number (ROI ID)assigned to each of the regions or may use the region number (ROI ID)assigned to each of regions instead of priority. The priority settingunit 322 may store the priority in the storage unit in association withthe region or may store the region number assigned to each of regions inthe storage unit in association with the region.

The priority is an identifier of each of regions (ROI) and isdiscrimination information capable of discriminating which of aplurality of regions in the captured image has undergone omission of theoverlapping region ROO. For example, of the two regions each includingthe overlapping region ROO, the priority setting unit 322 gives priority1 to one region and priority 2 to the other region. In this case, forexample, omission of the overlapping region ROO is performed onto theregion having a larger priority value. Note that the priority settingunit 322 may assign the same number as the region number assigned toeach of regions as a priority to the region. The priority setting unit322 stores the priority assigned to each of regions in the storage unitin association with the region image, for example.

The image processing controller 326 has a function of performing aprocess related to the transmission method according to the presentembodiment and thus transfers region information concerning the regionset for the image to the encoder 324 and the EBD buffer 310. Inaddition, the image processing controller 326 may set the additionalinformation other than the region information and may transfer theregion information and the other additional information to the EBDbuffer 310 as a series of additional information. Note that examples ofthe series of additional information including the region informationincludes information defined in Embedded Data described with referenceto FIG. 22, for example.

An example of the process of setting additional information can be aprocess of generating additional information. Examples of the process ofgenerating additional information include one or two or more of theprocesses, namely, a process of generating information indicating theamount of data in the region, a process of generating informationindicating the size of the region, and a process of generatinginformation indicating the priority of the region. As a specificexample, the image processing controller 326 may execute the variousprocesses for setting the additional information described above basedon the result of the processing by the region analyzer 318, the overlapdetector 320, and the priority setting unit 322 (that is, the result ofthe processes stored in the storage unit).

Note that the process of setting additional information is not limitedto the process of generating the additional information. For example,the image processing controller 326 may set information acquired fromthe image sensor device 102, such as exposure information and gaininformation, as additional information. Furthermore, the imageprocessing controller 326 may set, as additional information, dataconcerning various regions, such as data indicating a physical regionlength, data indicating an output region length, data indicating animage format, and data indicating the total amount of data.

The image processing controller 326 specifies a region included in eachof lines of an image on the basis of region designation informationacquired from the region cutout unit 316, the result of the processingby the region analyzer 318, the overlap detector 320, and the prioritysetting unit 322 (that is, result of the processing stored in thestorage unit). In addition, as another example, the image processingcontroller 326 may specify the region included in each of lines of theimage on the basis of region designation information (not illustrated)acquired from an external device. Subsequently, the image processingcontroller 326 sets the region information based on the specifiedregion. The image processing controller 326 then transfers the regioninformation set for each of regions to the encoder 354.

In addition, in a case where the region designation information is notacquired, the image processing controller 326 would not set the regioninformation.

Furthermore, the image processing controller 326 may generate frameinformation and transfer the generated frame information to the LINK/PHYcontroller 304, for example. The frame information is similar to theexample described above in the first transmission method.

The encoder 324 encodes the image data transferred from the image sensordevice 102 by a predetermined method such as a method corresponding tothe JPEG standard, for example.

In a case where the region information is not acquired from the imageprocessing controller 326, the encoder 324 transfers the encoded imagedata (that is, ordinary data) to the image data buffer 312.

In addition, in a case where the region information is acquired from theimage processing controller 326, the encoder 324 transfers the encodedregion data corresponding to the region to the image data buffer 312according to the acquired region information.

The image processing unit 302 includes a region cutout unit 316, aregion analyzer 318, an overlap detector 320, a priority setting unit322, an encoder 324, and an image processing controller 326, forexample, and thereby performs processing according to the transmissionmethod of the present embodiment. Note that the functional blocks of theimage processing unit 302 illustrated in FIG. 26 are obtained bydividing the functions of the image processing unit 106 for convenience,and how to divide the functions in the image processing unit 302 is notlimited to the example illustrated in FIG. 26.

The LINK/PHY controller 304 transfers frame information to the controlcode generator 306 and the PH generator 308 for each of lines, forexample. Furthermore, at this time, the LINK/PHY controller 304 maycontrol the protocol and the transmission path related to the datatransmission and may transfer a result of the control to the controlcode generator 306 and the PH generator 308.

The control code generator 306 sets an error correction code for each oflines. The control code generator 306 generates an error correction codefor a line on the basis of data for the line in the frame information.The control code generator 306 transfers the generated error correctioncode to the combining unit 314, for example. Furthermore, the controlcode generator 306 may generate an error correction code in cooperationwith the PH generator 308.

The PH generator 308 uses the frame information to generate a packetheader for each of lines. At this time, in the case of transmittingregion data, the PH generator 112 sets information indicating that theregion information (for example, region data) is to be transmitted asthe header information type to the extension region of the packet headeras described above. Thereafter, the PH generator 308 sets informationindicating that the region data is to be transmitted using the payload,for at least a part of the extension region.

The EBD buffer 310 is a buffer that temporarily holds additionalinformation transferred from the image processing unit 302 (the imageprocessing controller 326 in the example of FIG. 26). The EBD buffer 310outputs additional information to the combining unit 314 as “EmbeddedData” at a predetermined timing. The “Embedded Data” output from the EBDbuffer 310 may be transferred to the combining unit 314 via the imagedata buffer 312 described below.

The image data buffer 312 is a buffer that temporarily holds data(ordinary data or region data) transferred from the image processingunit 302 (encoder 324 in the example of FIG. 26). The image data buffer312 outputs the held data to the combining unit 314 for each of lines ata predetermined timing.

The combining unit 314 generates a packet to be transmitted, based onthe data acquired individually from the control code generator 306, thePH generator 308, the EBD buffer 310, and the image data buffer 312, forexample.

The transmitting unit 328 transmits the packet transferred from thecombining unit 314 for each of lines via the data bus B1.

For example, in a case where region 1, region 2, and region 3illustrated in FIG. 8 are set, the transmitting unit 328 transmits theregion information concerning each of regions as a part of additionalinformation (Embedded Data) using packet A2 while transmitting regiondata corresponding to each of regions using packet A1 for each of linesas illustrated in FIG. 24. Moreover, as another example, in a case whereregion 1, region 2, and region 3 illustrated in FIG. 10 are set, thetransmitting unit 328 transmits the region information concerning eachof regions as a part of additional information (Embedded Data) usingpacket A2 while transmitting region data corresponding to each ofregions using packet A1 for each of lines as illustrated in FIG. 25.

In addition, in a case where there is no region setting, that is, whereordinary data is output from the image data buffer 312, the transmittingunit 328 transmits, for each of lines, a packet having datacorresponding to each of lines being stored in the payload. Even in thiscase, the transmitting unit 328 can also transmit the additionalinformation as “Embedded Data”.

The above-described configuration is merely an example, and thefunctional configuration of the image sensor 100 is not limited to this.For example, the device or electronic device that executes each of theabove functions is not limited as long as each of the functions can berealized. As a more specific example, some of the functions of the ICchip 300 may be provided outside the IC chip 300.

Furthermore, additional functions may be added as long as the functionsof the image sensor 100 described above are not hindered, similarly tothe above-described first transmission method.

Hereinabove, an example of the functional configuration of the imagesensor 100 (transmission device) in the case of applying the thirdtransmission method has been described with reference to FIG. 26.

Configuration Example of Processor 200 (Reception Device)

Next, an example of the functional configuration of the processor 200(reception device) in the case of applying the third transmission methodwill be described. For example, FIG. 27 is a block diagram illustratingan example of the functional configuration of the processor 200 when thethird transmission method is applied. As illustrated in FIG. 27, theprocessor 200 includes, for example, a header separator 402, a headerinterpreter 404, a payload separator 406, an EBD interpreter 408, aregion data separator 410, and an image processing unit 412.

The header separator 402 separates the header data corresponding to aheader portion and the payload data corresponding to the payload portionfrom the received data. The header separator 402 separates the headerdata from the received data according to a rule defined in advance by astandard or the like, for example. Furthermore, the header separator 402may separate the payload data from the received data according to a rulepreliminarily defined by the standard, for example, or may separate thepayload data from the received data based on the result of theprocessing performed by the header interpreter 404.

The header interpreter 404 interprets the content indicated by theheader data.

As a specific example, the header interpreter 404 recognizes the formatof the information that is set in a region other than top three bits ofthe extension region according to the header information type set in thetop three bits of the extension region of a packet header. Subsequently,the header interpreter 404 reads out various types of information set inthe extension region according to the recognition result of the format.This enables the header interpreter 404 to recognize transmission ofregion (ROI) information (for example, region data) based on theinformation set in the extension region, for example. Subsequently, theheader interpreter 404 notifies the payload separator 406 and the imageprocessing unit 412 (an information extraction unit 414, which will bedescribed below) of the setting recognized according to the read resultof various types of information set in the extension region.Specifically, in a case where the header interpreter 404 has recognizedtransmission of the region (ROI) information (for example, region data),the header interpreter 404 notifies the payload separator 406 and theimage processing unit 412 of the recognition result.

The example of the process performed on the header interpreter 404 isnot limited to the above example. For example, the header interpreter404 may specify the position of the payload data and may transfer thespecified position to the header separator 402. The header interpreter404 can also distinguish between packet A2 that transmits “EmbeddedData” and packet A1 that transmits region data.

The payload separator 406 separates the additional information and theimage data (ordinary data or region data) from the payload data based onthe interpretation result from the header interpreter 404.

For example, in a case where the packet as a processing target is packetA2 or A3, the payload separator 406 may separate additional information(Embedded Data) from the packet.

Furthermore, as another example, in a case where the packet as aprocessing target is packet A1, the payload separator 406 separates theimage data from the payload data.

The payload separator 406 transmits additional information out ofvarious types of data separated from the payload data, to the EBDinterpreter 408. Furthermore, the payload separator 406 transmits imagedata (region data or ordinary data) out of various data separated fromthe payload data to the region data separator 410.

The EBD interpreter 238 interprets the content of additional information(Embedded Data) and then outputs the interpretation result of theadditional information to the region data separator 410 and the imageprocessing unit 412. The format of additional information (EmbeddedData) is as described above with reference to FIG. 22.

The region data separator 410 discriminates whether the image datastored in the payload is region data or ordinary data on the basis ofthe interpretation result of the additional information (Embedded Data).For example, in a case where region data is stored in the payload, theregion data separator 410 separates the region data from the payloaddata according to the interpretation result of the additionalinformation and then transmits the separated region data to the imageprocessing unit 412 (a region decoder 416, which will be describedbelow). In a case where ordinary data is stored in the payload, theregion data separator 410 separates the ordinary data from the payloaddata according to the interpretation result of the packet header andthen transmits the separated ordinary data to the image processing unit412 (an ordinary image decoder 420, which will be described below).

The image processing unit 412 includes an information extraction unit414, a region decoder 416, a region image generator 418, and an ordinaryimage decoder 420.

In a case where the information extraction unit 414 has recognizedtransmission of region (ROI) information based on the interpretationresult on the header interpreter 404, the information extraction unit414 extracts various types of information (particularly regioninformation) included in the additional information on the basis of theinterpretation result of the additional information (Embedded Data).Subsequently, the information extraction unit 214 outputs various typesof information extracted from the additional information to the regiondecoder 416 and the region image generator 418.

The region decoder 416 separates the region data corresponding to eachof regions from the series of region data transmitted from the regiondata separator 410 based on the interpretation result of the additionalinformation (Embedded Data). The region decoder 416 decodes theseparated region data of each of regions by a predetermined methodcorresponding to the encoding in the image sensor 100. At this time, theregion decoder 416 may change the content of the process based onvarious types of information extracted from the additional information(Embedded Data).

The region image generator 418 generates data representing an imagecorresponding to the region from region data for each of regions decodedby the region decoder 416, based on various types of information (forexample, region information) extracted from the additional information(Embedded Data).

For example, the region image generator 418 discriminates which regionincludes each of pieces of region data corresponding to partial regionfor each of lines, which is region data separated for each of regions,based on the region information included in the additional information(Embedded Data). Furthermore, the region image generator 418 recognizesthe position, within the image, of each of regions set in the image andthe size of the region based on the region information included in theadditional information (Embedded Data). Subsequently, the region imagegenerator 418 combines a series of region data corresponding to theregion based on the position in the image of each of regions set in theimage and based on the recognition result of the region size. Asdescribed above, the region image generator 418 combines the series ofregion data corresponding to each of regions to generate a region imagecorresponding to the region set in the image.

As described above, the region image generator 418 generates a regionimage for each of regions set in the image and outputs the generatedregion image to a predetermined output destination.

The ordinary image decoder 420 decodes the ordinary data transmittedfrom the region data separator 410 by a predetermined methodcorresponding to the encoding in the image sensor 100 to generate anordinary image and then outputs the generated ordinary image to apredetermined output destination.

The above-described configuration is merely an example, and thefunctional configuration of the processor 200 is not limited to this.For example, the device or electronic device that executes each of theabove functions is not limited as long as each of the functions can berealized. As a more specific example, some of the functions of theprocessor 200 may be provided outside the processor 200.

Furthermore, additional functions may be added as long as the functionsof the processor 200 described above are not hindered, similarly to theabove-described first transmission method.

(Evaluation)

As described above, in the third transmission method, the regioninformation of each of regions set in the image is transmitted in packetA2 as a part of additional information (Embedded Data), and the regiondata corresponding to each of regions is transmitted for each of linesin packet A1. Furthermore, in a case of transmitting the data of theregion set in the image, information indicating that the regioninformation (for example, region data) is to be transmitted is set inthe extension region of the header of each of packets. With the aboveconfiguration, the receiving side combines region data stored in thepayload of each of packets A1 based on the additional information(Embedded Data), making it possible to easily restore the region imageof the region set in the image. Furthermore, in the third transmissionmethod, the region-related information other than region data, such asthe coordinates of the regions in the first transmission method and thesecond transmission method, is not inserted in the payload. Therefore,the third transmission method makes it possible to further reduce theoverhead when transmitting region-related information (for example,region data), as compared with other transmission methods.

<3.4. Fourth Transmission Method>

Next, a fourth transmission method will be described. Similarly to theabove-described transmission methods, the image sensor 100 in the fourthtransmission method stores region data concerning the region set in theimage in the payload of the packet and transmits the data for each oflines. Therefore, in the following, the fourth transmission method willbe described focusing on the part different from the first transmissionmethod, and the detailed description of the parts that are substantiallysimilar to the first transmission method will be omitted.

(Data Format)

First, the format of data transmitted by the fourth transmission methodwill be described. For example, FIG. 28 is a diagram illustrating theformat of data transmitted by the fourth transmission method. Note that,in FIG. 28, the similar reference numerals as those in FIG. 5 denote thesimilar objects as those in the example illustrated in FIG. 5.

In the example illustrated in FIG. 28, at least a part of the series ofpackets A2 is used for transmitting Embedded Data. For example, EmbeddedData may be transmitted in a state of being stored in the payload ofpacket A2. Furthermore, as another example, the embedded data may betransmitted in a state of being stored in a region other than thepayload of packet A2. Note that the Embedded Data in the fourthtransmission method corresponds to additional information (in otherwords, information embedded by the image sensor 100) additionallytransmitted by the image sensor 100, similarly to the first transmissionmethod.

For example, FIG. 29 is a diagram illustrating an example of informationincluded in Embedded Data in the fourth transmission method. Inaddition, as illustrated in FIG. 29, each of items, the “ROI ID”, “upperleft coordinate”, “height”, “width”, “AD word length (AD bit)”,“exposure”, “gain”, and “sensing information” illustrates informationsimilar to the example illustrated in FIG. 6.

In FIG. 28, “RI” represents region information (ROI Information)regarding the partial region corresponding to each of regions datastored in the payload. For example, “RI” can include the number ofpartial regions included in the corresponding line or at least a part ofinformation of various types of information (for example, identificationinformation, coordinates, length, or the like) regarding each of partialregions. Note that, hereinafter, the information indicated by “RI” inFIG. 28 is also referred to as “information regarding partial regions”.

Information regarding the partial region is stored at the top of thepayload of packet A1. In addition, information having no change out ofother information excluding Y coordinate of each of partial regionsamong information included in the information regarding partial regionsbetween continuously transmitted packets A1 may be omitted in packet A1to be transmitted later. Furthermore, in a case where Y coordinate isincremented as 1 between continuously transmitted packets A1, the Ycoordinate may be omitted in packet A1 to be transmitted later. Inaddition, in a case where there is no change in any of the informationother than the Y coordinate of each of partial regions out ofinformation regarding the partial region between the continuouslytransmitted packets A1 and where the Y coordinate is incremented by 1,even the information regarding the partial region may be omitted inpacket A1 transmitted later. Note that this control will be describedbelow separately with a specific example.

Next, with reference to FIG. 30, an example of a configuration of thepacket header of packet A1 used for transmitting the region dataconcerning the region (ROI) set in the image will be described,particularly focusing on the configuration of the extension region. FIG.30 is a diagram illustrating an example of a configuration of the packetheader in the fourth transmission method.

As illustrated in FIG. 30, in the case of transmitting the region dataof the region (ROI) set in the image in the fourth transmission method,information indicating that the region information is to be transmitted(that is, the information corresponding to the type assuming thetransmission of the region information) is set as a header informationtype in the packet header of packet A1 used for transmitting the regiondata. Furthermore, information indicating the type according to the typeof information related to the partial region to be transmitted using thepayload (hereinafter, also referred to as “region type (ROI Info Type)”)is set for at least a part of the extension region. That is, theinformation indicating the region type (ROI Info Type) corresponds to anexample of “information indicating the type of information regarding thepartial region”. An example of the information indicating the regiontype will be described below. In addition, in the case of transmittingthe region data concerning the region (ROI) set in the image, thepayload length of packet A1 can change according to the horizontal widthof the region. Therefore, information indicating the payload length maybe set in a part of the extension region, similarly to the exampledescribed with reference to FIG. 4.

Subsequently, an example of information indicating a region type (ROIInfo Type) will be described with reference to FIG. 31. FIG. 31 is adiagram illustrating an example of information indicating a region type(ROI Info Type). As described above, there is a case, in the fourthtransmission method, where the information regarding the partial regionis inserted at the top of the payload. The type of various types ofinformation included as the information regarding the partial regionvaries depending on the difference of setting of the partial regionbetween the continuously transmitted packets A1. Therefore, in thepacket header, information indicating the region type is set accordingto the type of information inserted as the information on the partialregion at the top of the payload. Therefore, as illustrated in FIG. 31,the following will individually describe an example of a value that canbe set as the information indicating the region type and an example ofinformation that is transmitted as the information indicating the regiontype in the case where the value is set.

The value “4'b0000” is set in a case where there is no change in any ofthe information other than the Y coordinate of each of partial regionsout of the information regarding the partial region and where the Ycoordinate is incremented by 1. That is, in this case, the insertion ofthe information regarding the partial region in the payload is omitted.

The value “4'b0100” is set in a case where the number of partial regionsincluded in the corresponding line and the identification information ofeach of the partial regions are transmitted as the information regardingthe partial regions. In this case, as information regarding the partialregion, the number of partial regions (Num of ROI) included in the linecorresponding to the top is stored, and thereafter, the identificationinformation (ROI ID) for each of partial regions is stored in the orderin which each of the partial regions have been set to the correspondingline.

The value “4'b0101” is set in a case where the number of partial regionsincluded in the corresponding line, and the identification informationand a width (length in horizontal direction) of each of the partialregions, are transmitted as the information regarding the partialregions. In this case, as information regarding the partial region, thenumber of partial regions (Num of ROI) included in the linecorresponding to the top is stored, and thereafter, the identificationinformation (ROI ID) and the width (ROI LEN) for each of partial regionsis stored in the order in which each of the partial regions have beenset to the corresponding line.

The value “4'b0110” is set in a case where the number of partial regionsincluded in the corresponding line, and the identification informationand X coordinate (position in horizontal direction) of each of thepartial regions, are transmitted as the information regarding thepartial regions. In this case, as information regarding the partialregion, the number of partial regions (Num of ROI) included in the linecorresponding to the top is stored, and thereafter, the identificationinformation (ROI ID) and the X coordinate (ROI X) for each of partialregions is stored in the order in which each of the partial regions havebeen set to the corresponding line.

The value “4'b0111” is set in a case where the number of partial regionsincluded in the corresponding line, and the identification information,X coordinate (position in horizontal direction), and a width (length inhorizontal direction) of each of the partial regions, are transmitted asthe information regarding the partial regions. In this case, asinformation regarding the partial region, the number of partial regions(Num of ROI) included in the line corresponding to the top is stored,and thereafter, the identification information (ROI ID), X coordinate(ROI X) and the width (ROI LEN) for each of partial regions is stored inorder in which each of the partial regions have been set to thecorresponding line.

The value “4'b1000” is set in a case where the Y coordinate of a partialregion included in the corresponding line (in other words, Y coordinateof the line) is transmitted as the information regarding the partialregions. In this case, the Y coordinate (Y) of the line corresponding tothe top is stored as the information regarding the partial region.

The value “4'b1100” is set in a case where the Y coordinate of a partialregion included in the corresponding line (in other words, Y coordinateof the line), the number of partial regions included in the line, andidentification information of each of the partial regions aretransmitted as the information regarding the partial regions. In thiscase, as information regarding the partial region, the Y coordinate (Y)of the line corresponding to the top and the number of partial regions(Num of ROI) included in the line are stored in this order, andthereafter, the identification information (ROI ID) for each of partialregions is stored in the order in which each of the partial regions havebeen set to the corresponding line.

The value “4'b1101” is set in a case where the Y coordinate of a partialregion included in the corresponding line (in other words, Y coordinateof the line), the number of partial regions included in the line,identification information and a width (length in horizontal direction)of each of the partial regions are transmitted as the informationregarding the partial regions. In this case, as information regardingthe partial region, the Y coordinate (Y) of the line corresponding tothe top and the number of partial regions (Num of ROI) included in theline are stored in this order, and thereafter, the identificationinformation (ROI ID) and the width (ROI LEN) for each of partial regionsis stored in the order in which each of the partial regions have beenset to the corresponding line.

The value “4'b1110” is set in a case where the Y coordinate of a partialregion included in the corresponding line (in other words, Y coordinateof the line), the number of partial regions included in the line,identification information and X coordinate (position in horizontaldirection) of each of the partial regions are transmitted as theinformation regarding the partial regions. In this case, as informationregarding the partial region, the Y coordinate (Y) of the linecorresponding to the top and the number of partial regions (Num of ROI)included in the line are stored in this order, and thereafter, theidentification information (ROI ID) and the X coordinate (ROI X) foreach of partial regions are stored in the order in which each of thepartial regions have been set to the corresponding line.

The value “4'b1111” is set in a case where the Y coordinate of a partialregion included in the corresponding line (in other words, Y coordinateof the line), the number of partial regions included in the line, andidentification information, X coordinate (position in horizontaldirection), and a width (length in horizontal direction) of each of thepartial regions, are transmitted as the information regarding thepartial regions. In this case, as information regarding the partialregion, the Y coordinate (Y) of the line corresponding to the top andthe number of partial regions (Num of ROI) included in the line arestored in this order, and thereafter, the identification information(ROI ID), X coordinate (ROI X), and the width (ROI LEN) for each ofpartial regions is stored in the order in which each of the partialregions have been set to the corresponding line.

Example of Data

Next, with reference to FIG. 32 to FIG. 35, a specific example of datatransmitted by the fourth transmission method will be described as anexample in a case where the fourth transmission method is applied.

For example, FIG. 32 is a diagram illustrating an example of a region(ROI) that is set for an image. In FIG. 32, three regions, region 1,region 2, and region 3, are illustrated as an example of regions set foran image. Furthermore, in FIG. 32, a grid divided by broken linesextending in the vertical direction and the horizontal directionschematically illustrates a piece of unit data (for example, pixel)forming the image. In the example illustrated in FIG. 32, thecoordinates of the upper left vertex of each of grids are defined as thecoordinates of the grid, for convenience.

Furthermore, FIG. 33 is a diagram illustrating an example of datatransmitted by the fourth transmission method, illustrating one-framedata in a case where the region data concerning each of the regionsillustrated in FIG. 32 is transmitted by the fourth transmission method.Note that the configurations of packets A2 and A3 in FIG. 33 aresubstantially similar to the example described with reference to FIG. 5,and hence the following description will focus on the configuration ofpacket A1. Furthermore, each of “1”, “2”, and “3” illustrated in FIG. 33corresponds to the data of region 1, the data of region 2, and the dataof region 3 stored in the payload of packet A1.

For example, only the partial region of region 1 is set in the firstline of region 1. Therefore, region data of the partial region of region1 is stored in the payload of packet A1, for this line. Furthermore, atthe top of the payload, Y coordinate (Y) of the partial region, thenumber of partial regions (Num of ROI), and identification information(I1), X coordinate (X1), and a width (L1) of region 1, are inserted asinformation regarding the partial region. Therefore, “Fh (=4'b1111)” isset as the region type (ROI Info Type) in the packet header of packetA1. Furthermore, regarding the second line to the fourth line of region1 to which the region data is transmitted next, there is no change fromthe first line of region 1 except for the Y coordinate (Y) of thepartial region in the information regarding the partial region.Therefore, in the payload of packet A1 used for transmitting the data ofthe partial regions in the second to fourth lines of region 1,information regarding the partial regions is not inserted at the top.Therefore, “0h (=4'b0000)” is set as the region type (ROI Info Type) inthe packet header of packet A1.

In addition, the partial region of region 1 and the partial region ofregion 2 are set in the line corresponding to the fifth line of region1. Therefore, for this line, the payload of packet A1 stores the regiondata of the partial region of region 1 and the region data of thepartial region of region 2. Furthermore, at the top of the payload, Ycoordinate (Y) of the partial region, the number of partial regions (Numof ROI), identification information (I1), X coordinate (X1), and thewidth (L1) of region 1, identification information (I2), X coordinate(X2), and a width (L2) of region 2, are inserted as informationregarding the partial region. Therefore, “Fh (=4'b1111)” is set as theregion type (ROI Info Type) in the packet header of packet A1.Furthermore, in the line corresponding to the sixth line of region 1 forwhich the region data is transmitted next, the information of thepartial region corresponding to region 2 has been changed from the fifthline, that is, two independent regions are set as the partial regions ofregion 2. Therefore, for this line, the payload of packet A1 stores theregion data of the partial region of region 1 and the region data of thetwo partial regions of region 2. Furthermore, at the top of the payload,Y coordinate (Y) of the partial region, the number of partial regions(Num of ROI), identification information (I1), X coordinate (X1), andthe width (L1) of region 1, and identification information (I2), Xcoordinate (X2), and the width (L2) of the two partial regions of region2, are inserted as information regarding the partial region. Therefore,“Fh (=4'b1111)” is set as the region type (ROI Info Type) in the packetheader of packet A1.

In addition, only the partial region of region 1 is set in the linecorresponding to the seventh line of region 1. Therefore, region data ofthe partial region of region 1 is stored in the payload of packet A1,for this line. Note that the line is similar to the immediatelypreceding line except that the partial region of region 2 does notexist, that is, the number of regions (Num of ROI) and theidentification information (I1) of region 1 are inserted at the top ofthe payload, as information regarding the partial region. Therefore, “4h(=4'b0100)” is set as the region type (ROI Info Type) in the packetheader of packet A1.

Furthermore, region 3 is set to include three line regions in thehorizontal direction, in which the X coordinate of the left end of thepartial region corresponding to the first and third lines differs fromthe X coordinate of the left end of the partial region corresponding tothe second line.

Specifically, only the partial region of region 3 is set in the firstline of region 3. Therefore, region data of the partial region of region3 is stored in the payload of packet A1, for this line. Furthermore, atthe top of the payload, Y coordinate (Y) of the partial region and thenumber of partial regions (Num of ROI), and identification information(I3), X coordinate (X3), and a width (L3) of region 3, are inserted asinformation regarding the partial region. Therefore, “Fh (=4'b1111)” isset as the region type (ROI Info Type) in the packet header of packetA1. The second line of region 3 is similar to the immediately precedingline except that the X coordinate and the width of the partial regionare different, that is, the number of regions (Num of ROI), and theidentification information (I3), the X coordinate (X3), and the width(L3) of region 3, are inserted at the top of the payload, as informationregarding the partial region. Therefore, “7h (=4'b0111)” is set as theregion type (ROI Info Type) in the packet header of packet A1.Similarly, the third line of region 3 is similar to the immediatelypreceding line except that the X coordinate and the width of the partialregion are different, that is, the number of regions (Num of ROI), andthe identification information (I3), the X coordinate (X3), and thewidth (L3) of region 3, are inserted at the top of the payload, asinformation regarding the partial region. Therefore, “7h (=4'b0111)” isset as the region type (ROI Info Type) in the packet header of packetA1.

In addition, FIG. 34 is a diagram illustrating another example of aregion (ROI) that is set for an image. In FIG. 34, three regions, region1, region 2, and region 3, are illustrated as an example of regions setfor an image. Note that, in FIG. 34, a grid divided by broken linesextending in the vertical direction and the horizontal directionschematically illustrates a piece of unit data forming the image,similarly to the example described with reference to FIG. 8. Therefore,also in the example illustrated in FIG. 34, the coordinates of the upperleft vertex of each of grids are defined as the coordinates of the grid,for convenience.

Furthermore, FIG. 35 is a diagram illustrating another example of datatransmitted by the fourth transmission method, illustrating one-framedata in a case where the region data concerning each of the regionsillustrated in FIG. 34 is transmitted by the fourth transmission method.Note that the configurations of packets A2 and A3 in FIG. 35 as well aresubstantially similar to the example described with reference to FIG. 5,and hence the description will focus on the configuration of packet A1.Furthermore, each of “1”, “2”, and “3” illustrated in FIG. 35corresponds to the data of region 1, the data of region 2, and the dataof region 3 stored in the payload of packet A1.

For example, only the partial region of region 1 is set in the firstline of region 1. Therefore, region data of the partial region of region1 is stored in the payload of packet A1, for this line. Furthermore, atthe top of the payload, Y coordinate (Y) of the partial region, thenumber of partial regions (Num of ROI), and identification information(I1), X coordinate (X1), and a width (L1) of region 1, are inserted asinformation regarding the partial region. Therefore, “Fh (=4'b1111)” isset as the region type (ROI Info Type) in the packet header of packetA1. Furthermore, regarding the second line to the fourth line of region1 to which the region data is transmitted next, there is no change fromthe first line of region 1 except for the Y coordinate (Y) of thepartial region in the information regarding the partial region.Therefore, at the top of the payload of packet A1 used for transmittingthe data of the partial regions in the second to fourth lines of region1, information regarding the partial regions is not inserted. Therefore,“0h (=4'b0000)” is set as the region type (ROI Info Type) in the packetheader of packet A1.

In addition, the partial region of region 1 and the partial region ofregion 2 are set in the line corresponding to the fifth line of region1. Therefore, for this line, the payload of packet A1 stores the regiondata of the partial region of region 1 and the region data of thepartial region of region 2. Furthermore, at the top of the payload, Ycoordinate (Y) of the partial region, the number of partial regions (Numof ROI), identification information (I1), X coordinate (X1), and thewidth (L1) of region 1, identification information (I2), X coordinate(X2), and a width (L2) of region 2, are inserted as informationregarding the partial region. Therefore, “Fh (=4'b1111)” is set as theregion type (ROI Info Type) in the packet header of packet A1. Inaddition, in the line, a part of the partial region of region 1 and apart of the partial region of region 2 overlap each other. Regarding theoverlapping region (overlapping region ROO), which of the region data ofthe partial region of region 1 and the region data of the partial regionof region 2 is stored in the payload (that is, which is omitted) isoptional. For example, in the example illustrated in FIG. 35, theoverlapping region (overlapping region ROO) with the first region isomitted for the second region. Even in a case where the overlappingregion is omitted, the X coordinate and the width of each of partialregions set as information regarding the partial region are set based onthe state before the omission.

In addition, only the partial region of region 2 is set in the linecorresponding to the second line of region 2. Therefore, region data ofthe partial region of region 2 is stored in the payload of packet A1,for this line. Note that the line is similar to the immediatelypreceding line except that the partial region of region 1 does notexist, that is, the number of regions (Num of ROI) and theidentification information (I2) of region 2 are inserted at the top ofthe payload, as information regarding the partial region. Therefore, “4h(=4'b0100)” is set as the region type (ROI Info Type) in the packetheader of packet A1. Furthermore, regarding the third line of region 2for which the region data is transmitted next, there is no change fromthe second line of region 2 except for the Y coordinate (Y) of thepartial region in the information regarding the partial region.Therefore, at the top of the payload of packet A1 used for transmittingthe data of the partial regions in the third lines of region 2,information regarding the partial regions is not inserted. Therefore,“0h (=4'b0000)” is set as the region type (ROI Info Type) in the packetheader of packet A1.

The region 3 is similar to the example described with reference to FIGS.32 and 33 and thus detailed description thereof will be omitted.

Example of Configuration of Image Sensor 100 (Transmission Device)

Next, an example of a functional configuration of the image sensor 100(transmission device) in the case of applying the fourth transmissionmethod will be described. For example, FIG. 36 is a block diagramillustrating an example of a functional configuration of the imagesensor 100 in the case of applying the fourth transmission method. Asillustrated in FIG. 36, the image sensor 100 includes an image sensordevice 102 and an IC chip 330, for example. The image sensor 100 in FIG.36 operates on electric power supplied from an internal power supply(not illustrated) such as a battery included in the communication system1000, or electric power supplied from an external power supply of thecommunication system 1000.

The image sensor device 102 illustrated in FIG. 36 is substantiallysimilar to the image sensor device 102 illustrated in FIG. 12.

The IC chip 330 is a chip formed with an IC in which a circuit relatedto the data transmission function using the third transmission method isintegrated and is used to process image data transferred from the imagesensor device 102 and transmit the data corresponding to the generatedimage. The circuit related to the data transmission function accordingto the fourth transmission method is not limited to the implementationin the form of one IC chip, and may be formed of a plurality of ICchips.

The IC chip 330 includes an image processing unit 332, a LINK/PHYcontroller 334, a control code generator 336, a PH generator 338, an EBDbuffer 340, an image data buffer 342, a combining unit 344, and atransmitting unit 358, for example.

The image processing unit 332 can be formed as one circuit having afunction of performing processes related to the transmission methodaccording to the present embodiment. When performing the process relatedto the transmission method according to the present embodiment, theimage processing unit 332 controls, for each of lines in the image, totransmit region data corresponding to the region set in the image,through the LINK/PHY controller 334, the control code generator 336, thePH generator 338, the EBD buffer 340, the image data buffer 342, thecombining unit 344, and the transmitting unit 358, using the thirdtransmission method. The image processing unit 332 can also control totransmit the image data transferred from the image sensor device 102(that is, the data representing the entire image) for each of lines.

The image processing unit 332 can be formed by a processor such as anMPU, for example.

The function of the image processing unit 332 will be described in theform of functional blocks. As illustrated in FIG. 36, the imageprocessing unit 332 includes a region cutout unit 346, a region analyzer348, an overlap detector 350, a priority setting unit 322, an imageprocessing controller 326, and an encoder 324, for example. Note thatthe region cutout unit 346, the region analyzer 348, the overlapdetector 350, and the priority setting unit 322 are respectivelysubstantially similar to the region cutout unit 316, the region analyzer318, the overlap detector 320, and the priority setting unit 322illustrated in FIG. 26, and thus, detailed description thereof will beomitted.

The image processing controller 356 has a function of performing aprocess related to the transmission method according to the presentembodiment and thus transfers region information concerning the regionset for the image to the encoder 324 and the EBD buffer 340. Inaddition, the image processing controller 356 may set the additionalinformation other than the region information and may transfer theregion information and the other additional information to the EBDbuffer 340 as a series of additional information. Note that examples ofthe series of additional information including the region informationincludes information defined in Embedded Data described with referenceto FIG. 29, for example.

An example of the process of setting additional information can be aprocess of generating additional information. Examples of the process ofgenerating additional information include one or two or more of theprocesses, namely, a process of generating information indicating theamount of data in the region, a process of generating informationindicating the size of the region, and a process of generatinginformation indicating the priority of the region. As a specificexample, the image processing controller 356 may execute the variousprocesses for setting the additional information described above basedon the result of the processing by the region analyzer 348, the overlapdetector 350, and the priority setting unit 322 (that is, the result ofthe processing stored in the storage unit).

Note that the process of setting additional information is not limitedto the process of generating the additional information. For example,the image processing controller 356 may set information acquired fromthe image sensor device 102, such as exposure information and gaininformation, as additional information. Furthermore, the imageprocessing controller 356 may set, as additional information, dataconcerning various regions, such as data indicating a physical regionlength, data indicating an output region length, data indicating animage format, and data indicating the total amount of data.

The image processing controller 356 specifies a region included in eachof lines of an image based on region designation information acquiredfrom the region cutout unit 346, the result of the processing by theregion analyzer 348, the overlap detector 350, and the priority settingunit 322 (that is, result of the processing stored in the storage unit).In addition, as another example, the image processing controller 356 mayspecify the region included in each of lines of the image based onregion designation information (not illustrated) acquired from anexternal device. Subsequently, the image processing controller 356 setsthe region information based on the specified region. The imageprocessing controller 356 then transfers the region information set foreach of regions to the encoder 354.

In addition, in a case where the region designation information is notacquired, the image processing controller 356 would not set the regioninformation.

Furthermore, the image processing controller 356 may generate frameinformation and transfer the generated frame information to the LINK/PHYcontroller 334, for example. The frame information is similar to theexample described above in the first transmission method.

The encoder 354 encodes the image data transferred from the image sensordevice 102 by a predetermined method such as a method corresponding tothe JPEG standard, for example.

In a case where the region information is not acquired from the imageprocessing controller 356, the encoder 354 transfers the encoded imagedata (that is, ordinary data) to the image data buffer 342.

In addition, in a case where the region information is acquired from theimage processing controller 356, the encoder 354 transfers the acquiredregion information and the encoded region data corresponding to theregion to the image data buffer 342.

The image processing unit 332 includes a region cutout unit 346, aregion analyzer 348, an overlap detector 350, a priority setting unit352, an encoder 354, and an image processing controller 356, forexample, and thereby performs processing according to the transmissionmethod of the present embodiment. Note that the functional blocks of theimage processing unit 332 illustrated in FIG. 36 are obtained bydividing the functions of the image processing unit 332 for convenience,and how to divide the functions in the image processing unit 332 is notlimited to the example illustrated in FIG. 36.

The LINK/PHY controller 334 transfers frame information to the controlcode generator 336 and the PH generator 338 for each of lines, forexample. Furthermore, at this time, the LINK/PHY controller 334 maycontrol the protocol and the transmission path related to the datatransmission and may transfer a result of the control to the controlcode generator 336 and the PH generator 338.

The control code generator 336 sets an error correction code for each oflines. The control code generator 336 generates an error correction codefor a line on the basis of data for the line in the frame information.The control code generator 336 transfers the generated error correctioncode to the combining unit 344, for example. Furthermore, the controlcode generator 336 may generate an error correction code in cooperationwith the PH generator 338.

The PH generator 338 uses the frame information to generate a packetheader for each of lines. At this time, in the case of transmittingregion data, the PH generator 112 sets information indicating that theregion information (for example, region data) is to be transmitted asthe header information type to the extension region of the packet headeras described above. Thereafter, the PH generator 338 sets, for at leasta part of the extension region, information indicating that the regiondata is to be transmitted using the payload and information indicatingthe region type (ROI Info Type) corresponding to the informationregarding the partial region to be inserted into the payload.

The EBD buffer 340 is a buffer that temporarily holds additionalinformation transferred from the image processing unit 332 (the imageprocessing controller 356 in the example of FIG. 36). The EBD buffer 340outputs additional information to the combining unit 344 as “EmbeddedData” at a predetermined timing. The “Embedded Data” output from the EBDbuffer 340 may be transferred to the combining unit 344 via the imagedata buffer 342 described below.

The image data buffer 342 is a buffer that temporarily holds data(ordinary data or region data) transferred from the image processingunit 332 (encoder 324 in the example of FIG. 36). The image data buffer342 outputs the held data to the combining unit 344 for each of lines ata predetermined timing.

The combining unit 344 generates a packet to be transmitted based on thedata acquired from the control code generator 336, the PH generator 338,the EBD buffer 340, and the image data buffer 342, for example.

The transmitting unit 358 transmits the packet transferred from thecombining unit 344 for each of lines via the data bus B1.

For example, in a case where region 1, region 2, and region 3illustrated in FIG. 32 are set, the transmitting unit 358 transmits theregion information concerning each of regions as a part of additionalinformation (Embedded Data) using packet A2 while transmitting regiondata corresponding to each of regions using packet A1 for each of linesas illustrated in FIG. 33. Moreover, as another example, in a case whereregion 1, region 2, and region 3 illustrated in FIG. 34 are set, thetransmitting unit 358 transmits the region information concerning eachof regions as a part of additional information (Embedded Data) usingpacket A2 while transmitting region data corresponding to each ofregions using packet A1 for each of lines as illustrated in FIG. 35.

In addition, in a case where there is no region setting, that is, whereordinary data is output from the image data buffer 342, the transmittingunit 358 transmits, for each of lines, a packet having datacorresponding to each of lines being stored in the payload. Even in thiscase, the transmitting unit 358 can also transmit the additionalinformation as “Embedded Data”.

The above-described configuration is merely an example, and thefunctional configuration of the image sensor 100 is not limited to this.For example, the device or electronic device that executes each of theabove functions is not limited as long as each of the functions can berealized. As a more specific example, some of the functions of the ICchip 330 may be provided outside the IC chip 330.

Furthermore, additional functions may be added as long as the functionsof the image sensor 100 described above are not hindered, similarly tothe above-described first transmission method.

Hereinabove, an example of the functional configuration of the imagesensor 100 (transmission device) in the case of applying the thirdtransmission method has been described with reference to FIG. 36.

Configuration Example of Processor 200 (Reception Device)

Next, an example of the functional configuration of the processor 200(reception device) in the case of applying the fourth transmissionmethod will be described. For example, FIG. 37 is a block diagramillustrating an example of the functional configuration of the processor200 when the fourth transmission method is applied. As illustrated inFIG. 37, the processor 200 includes, for example, a header separator432, a header interpreter 434, a payload separator 436, an EBDinterpreter 438, a region data separator 440, and an image processingunit 442.

The header separator 432 separates the header data corresponding to aheader portion and the payload data corresponding to the payload portionfrom the received data. The header separator 432 separates the headerdata from the received data according to a rule defined in advance by astandard or the like, for example. Furthermore, the header separator 432may separate the payload data from the received data according to a rulepreliminarily defined by the standard, for example, or may separate thepayload data from the received data based on the result of theprocessing performed by the header interpreter 434.

The header interpreter 434 interprets the content indicated by theheader data.

As a specific example, the header interpreter 434 recognizes the formatof the information that is set in a region other than top three bits ofthe extension region according to the header information type set in thetop three bits of the extension region of a packet header. Subsequently,the header interpreter 434 reads out various types of information set inthe extension region according to the recognition result of the format.This enables the header interpreter 434 to recognize transmission ofregion (ROI) information (for example, region data) or the region type,that is, the type of information to be inserted in the payload asinformation related to the partial region based on, for example, theinformation set in the extension region. Subsequently, the headerinterpreter 434 notifies the payload separator 436 and the imageprocessing unit 442 (an information extraction unit 444, which will bedescribed below) of the setting recognized according to the read resultof various types of information set in the extension region.Specifically, in a case where the header interpreter 434 has recognizedtransmission of the region (ROI) information (for example, region data),the header interpreter 434 notifies the payload separator 436 and theimage processing unit 442 of the recognition result and the recognizedregion type.

The example of the process performed on the header interpreter 434 isnot limited to the above example. For example, the header interpreter434 may specify the position of the payload data and may transfer thespecified position to the header separator 432. The header interpreter434 can also distinguish between packet A2 that transmits “EmbeddedData” and packet A1 that transmits region data.

The payload separator 436 separates the additional information,information regarding a partial region, and the image data (ordinarydata or region data) from the payload data based on the interpretationresult from the header interpreter 434.

For example, in a case where the packet as a processing target is packetA2 or A3, the payload separator 436 may separate additional information(Embedded Data) from the packet.

Furthermore, as another example, in a case where the packet as aprocessing target is packet A1, the payload separator 436 separatesinformation (ROI Information (RI)) regarding the partial region and theimage data from the payload data.

The payload separator 436 transmits additional information out ofvarious types of data separated from the payload data, to the EBDinterpreter 438. Furthermore, the payload separator 436 transmitsinformation regarding a partial region and image data (region data orordinary data) out of various data separated from the payload data, tothe region data separator 440. The payload separator 436.

The EBD interpreter 438 interprets the content of additional information(Embedded Data) and then outputs the interpretation result of theadditional information to the region data separator 440 and the imageprocessing unit 442. The format of the additional information (EmbeddedData) is as described above with reference to FIG. 29.

The region data separator 440 discriminates whether the image datastored in the payload is region data or ordinary data on the basis ofthe interpretation result of the additional information (Embedded Data).For example, in a case where the region data is stored in the payload,the region data separator 440 separates a series of data including theinformation regarding a partial region (ROI Information (RI)) and theregion data from the payload data according to the interpretation resultof the additional information and then transmits the separated data tothe image processing unit 442 (a region decoder 416, which will bedescribed below). In a case where ordinary data is stored in thepayload, the region data separator 440 separates the ordinary data fromthe payload data and then transmits the separated ordinary data to theimage processing unit 442 (an ordinary image decoder 450, which will bedescribed below).

The image processing unit 442 includes an information extraction unit444, a region decoder 446, a region image generator 448, and an ordinaryimage decoder 450.

In a case where the information extraction unit 444 has recognizedtransmission of region (ROI) information on the basis of theinterpretation result on the header interpreter 434, the informationextraction unit 444 extracts various types of information (particularlyregion information) included in the additional information on the basisof the interpretation result of the additional information (EmbeddedData). Subsequently, the information extraction unit 214 outputs varioustypes of information extracted from the additional information and theregion type acquired as the interpretation result obtained by the headerinterpreter 434 to the region decoder 446 and the region image generator448.

The region decoder 446 extracts information (ROI Information (RI))regarding the partial region from the series of data transmitted fromthe region data separator 440 according to the region type output fromthe information extraction unit 444. Furthermore, the region decoder 446separates region data corresponding to each of regions from the seriesof data based on the extracted information regarding the partial region.At this time, the region decoder 446 recognizes the number of pieces ofregion data included in the series of data (in other words, the numberof regions included in the corresponding line) in accordance with thenumber of partial regions (Num of ROI) included in the informationregarding the partial region. Furthermore, the region decoder 446recognizes the length of the region data corresponding to each ofregions in accordance with the width (ROI LEN) of each of regionsincluded in the information regarding the partial region. This enablesthe region decoder 446 to separate the region data corresponding to eachof regions from the series of data described above. The region decoder446 decodes the separated region data of each of regions by apredetermined method corresponding to the encoding in the image sensor100. At this time, the region decoder 446 may change the content of theprocess based on various types of information extracted from theadditional information (Embedded Data).

The region image generator 448 generates data representing an imagecorresponding to the region from region data for each of regions decodedby the region decoder 446, based on various types of information (forexample, region information) extracted from the additional information(Embedded Data) and based on information regarding a partial region (ROIInformation (RI)).

For example, the region image generator 448 discriminates which regionincludes each of pieces of region data corresponding to partial regionfor each of lines, which is region data separated for each of regions,based on the identification information (ROI ID) included in theinformation (ROI Information (RI)) regarding a partial region.Furthermore, the region image generator 448 recognizes the position,within the image, of each of regions set in the image and the size ofthe region based on X coordinate (ROI X) and a width (ROI LEN) includedin the information regarding a partial region. Subsequently, the regionimage generator 448 combines a series of region data corresponding tothe region based on the position in the image of each of regions set inthe image and based on the recognition result of the region size. Asdescribed above, the region image generator 448 combines the series ofregion data corresponding to each of regions to generate a region imagecorresponding to the region set in the image.

As described above, the region image generator 448 generates a regionimage for each of regions set in the image and outputs the generatedregion image to a predetermined output destination.

The ordinary image decoder 450 decodes the ordinary data transmittedfrom the region data separator 440 by a predetermined methodcorresponding to the encoding in the image sensor 100 to generate anordinary image and then outputs the generated ordinary image to apredetermined output destination.

The above-described configuration is merely an example, and thefunctional configuration of the processor 200 is not limited to this.For example, the device or electronic device that executes each of theabove functions is not limited as long as each of the functions can berealized. As a more specific example, some of the functions of theprocessor 200 may be provided outside the processor 200.

Furthermore, additional functions may be added as long as the functionsof the processor 200 described above are not hindered, similarly to theabove-described first transmission method.

(Evaluation)

As described above, in the fourth transmission method, the regioninformation of each of regions set in the image is transmitted in packetA2 as a part of additional information (Embedded Data), and the regiondata corresponding to each of regions is transmitted for each of linesin packet A1. Furthermore, in a case of transmitting the data of theregion set in the image, information indicating that the regioninformation (for example, region data) is to be transmitted is set inthe extension region of the header of each of packets. Furthermore, inthis case, information (ROI Information (RI)) regarding each of partialregions of each of regions included in the corresponding line can beinserted into the payload of each of packets. At this time, a regiontype (ROI Info Type) is set for the extension region of the header ofeach of packets in accordance with the type of information regarding apartial region inserted in the payload. With the above configuration,the receiving side combines region data stored in the payload of each ofpackets A1 based on the additional information (Embedded Data) andinformation (ROI Information (RI)) regarding a partial region, making itpossible to easily restore the region image of the region set in theimage. As described above, in the fourth transmission method, it ispossible to finely set information regarding each of partial regions foreach of lines of the region (ROI) set in the image as informationregarding the partial region (ROI Information (RI). Due to suchcharacteristics, the fourth method makes it possible to transmitinformation regarding a region having an arbitrary shape.

<3.5. Supplement>

Note that the example described above is merely an example, and does notnecessarily limit the technology according to the present disclosure,particularly the technology related to the transmission method fortransmitting the region data concerning the region (ROI) that is set foran image. For example, the transmission method is not limited as long asthe receiving side can recognize whether or not the payload of thepacket includes region data. There is no need to set informationindicating that the region information is to be transmitted to theextension region of the packet header. As a specific example, thereceiving side may recognize whether or not the payload of the packetincludes region data based on an extension region of a packet header orinformation regarding a region (ROI) included in additional information(Embedded Data), such as “ROI ID”, “upper left coordinate”, “height”,and “width”.

4. CONCLUSION

Preferred embodiments of the present disclosure have been described indetail hereinabove with reference to the accompanying drawings, but thetechnical scope of the present disclosure is not limited to suchexamples. It is obvious that a person with an ordinary skill in atechnological field of the present disclosure could conceive of variousalterations or corrections within the scope of the technical ideasdescribed in the appended claims, and it should be understood that suchalterations or corrections will naturally belong to the technical scopeof the present disclosure.

Furthermore, the advantageous effects described in the presentspecification are only descriptive or exemplary, and non-limiting. Inother words, the technology according to the present disclosure affords,in addition to or instead of the foregoing advantageous effects, otheradvantageous effects which are obvious to a person skilled in the artfrom the descriptions of the present specification.

Note that the following configurations also fall within the technicalscope of the present disclosure.

(1)

A transmission device comprising

an image processing unit that controls to transmit a first packetincluding region information regarding a region detected from an imageand controls to transmit a second packet for each of lines in which atleast the region has been detected in the image, the second packetincluding: a header including information regarding the line; and apayload including region data concerning a partial region correspondingto the line, out of the region,

wherein the header includes: identification information of the dataincluded in the payload; an error correction code of the informationincluded in the header; and an extension region provided so as to beinterposed between the identification information and the errorcorrection code, and

information indicating whether or not the region data is included in thepayload corresponding to the header is set in at least a part of theextension region.

(2)

The transmission device according to (1), wherein the header includes:information indicating a start of a frame, information indicating an endof the frame, information indicating whether or not a corresponding lineis valid, a line number, information indicating whether or not a linehas embedded data, the identification information, the extension region,and the error correction code, being arranged in this order.

(3)

The transmission device according to (1) or (3), wherein the secondpacket is individually transmitted for each of the partial regionsincluded in the line.

(4)

The transmission device according to (3),

wherein the payload includes information regarding coordinates of thecorresponding partial region, and

information indicating that the payload includes the informationregarding the coordinates is set in the extension region.

(5)

The transmission device according to (1) or (2),

wherein the second packet is transmitted for each of lines, and

the payload included in the second packet includes the region data foreach of the partial regions included in the line.

(6)

The transmission device according to (5), wherein the payload of thesecond packet corresponding to the line including a plurality of thepartial regions includes data corresponding to an interval between twopartial regions separated from each other in the image, between theregion data corresponding to each of the two partial regions.

(7)

The transmission device according to (6),

wherein the payload includes information regarding coordinates of a toppartial region out of the partial regions included in the correspondingline, and

information indicating that the payload includes information regardingthe coordinates is set in the extension region.

(8)

The transmission device according to (5), wherein a position, within theimage, of the partial region corresponding to each of one or more piecesof the region data included in the payload is specified based on theregion information regarding the region including the partial regionincluded in the first packet.

(9)

The transmission device according to (5),

wherein the payload includes information regarding the partial regionincluded in the corresponding line, and

information indicating type of information regarding the partial regionincluded in the payload is set in the extension region.

(10)

The transmission device according to (9), wherein the informationregarding the partial region includes information regarding the numberof the partial regions included in the corresponding line and includesat least one of identification information, coordinates, or lengthregarding the partial region for each of the partial regions.

(11)

The transmission device according to (9) or (10), wherein, in thepayload corresponding to a second line immediately succeeding a firstline, information having a set value different from the first line outof the information regarding the partial region is set.

(12)

The transmission device according to any one of (1) to (11), wherein theregion information includes at least one of identification information,coordinates, height, width, AD word length, exposure, gain, which arerelated to the region, information regarding a subject, or informationfor post-stage signal processing.

(13)

A reception device comprising

an image processing unit that restores a partial image corresponding toa region detected from an image, based on a result of reception of afirst packet including region information regarding the region and basedon a result of reception of a second packet for each of lines in whichat least the region has been detected in the image, the second packetincluding: a header including information regarding the line; and apayload including region data concerning a partial region correspondingto the line, out of the region,

wherein the header includes: identification information of the dataincluded in the payload; an error correction code of the informationincluded in the header; and an extension region provided so as to beinterposed between the identification information and the errorcorrection code, and

information indicating whether or not the region data is included in thepayload corresponding to the header is set in at least a part of theextension region.

(14)

The reception device according to (13),

wherein, in a case where information indicating that the region data isincluded in the payload corresponding to the header is set for at leasta part of the extension region of the header,

the image processing unit restores the partial image corresponding tothe region, based on the result of reception of the first packet and theresult of reception of the second packet.

(15)

The reception device according to (13) or (14), wherein the imageprocessing unit restores the partial image corresponding to the regionby performing restoration processing on the region data included in thepayload of one or more of the second packets, based on the regioninformation.

(16)

The reception device according to (13) or (14),

wherein the second packet is transmitted individually for each of thepartial regions included in the line,

the payload includes information regarding coordinates of thecorresponding partial region,

information indicating that the payload includes the informationregarding the coordinates is set in the extension region, and

the image processing unit performs restoration processing on the regiondata included in the payload of the one or more second packets andthereby restores the partial image corresponding to the region, for eachof the regions set in the image, based on the region information, andthe information regarding the coordinates included in the payload of theone or more second packets corresponding to the partial region of theregion.

(17)

The reception device according to (13) or (14),

wherein the payload of the second packet corresponding to the lineincluding a plurality of the partial regions includes data correspondingto an interval between two partial regions separated from each other inthe image, between the region data corresponding to each of the twopartial regions,

the payload includes information regarding coordinates of a top partialregion out of the partial regions included in the corresponding line,

Information indicating that the payload includes the informationregarding coordinates is set in the extension region, and

the image processing unit performs, for each of the regions, restorationprocessing on the region data included in the payload of the one or moresecond packets and thereby restores the partial image corresponding tothe region, based on the region information, and the coordinatesincluded in the payload of the one or more second packets.

(18)

The reception device according to (13) or (14),

wherein the payload includes information regarding the partial regionincluded in the corresponding line,

information indicating the type of information regarding the partialregion included in the payload is set in the extension region, and

the image processing unit performs, for each of the regions, restorationprocessing on the region data included in the payload of the one or moresecond packets and thereby restores the partial image corresponding tothe region, based on the region information and the informationregarding the partial region included in the payload which has beenextracted on the basis of the information indicating the type set in theextension region for each of the one or more second packets.

(19)

A communication system comprising:

a transmission device including a first image processing unit thatcontrols to transmit a first packet including region informationregarding a region detected from an image and controls to transmit asecond packet for each of lines in which at least the region has beendetected in the image, the second packet including a header includinginformation regarding the line, and a payload including region dataconcerning a partial region corresponding to the line, out of theregion; and

a reception device including a second image processing unit thatrestores a partial image corresponding to the region on the basis of aresult of reception of the first packet and a result of reception of thesecond packet for each of lines in which at least the region has beendetected in the image,

wherein the header includes: identification information of the dataincluded in the payload; an error correction code of the informationincluded in the header; and an extension region provided so as to beinterposed between the identification information and the errorcorrection code, and

information indicating whether or not the region data is included in thepayload corresponding to the header is set in at least a part of theextension region.

(20)

A transmission device comprising

an image processing unit that controls to transmit a first packetincluding region information regarding a region detected from an imageand controls to transmit a second packet for each of lines in which atleast the region has been detected in the image, the second packetincluding: a header including information regarding the line; and apayload including region data concerning a partial region correspondingto the line, out of the region,

wherein whether or not the payload of the second packet includes theregion data is recognized on the receiving side of the second packet,based on at least one of information included in the headercorresponding to the payload or the region information.

(21)

A reception device comprising

an image processing unit that restores a partial image corresponding toa region detected from an image, based on a result of reception of afirst packet including region information regarding the region and basedon a result of reception of a second packet for each of lines in whichat least the region has been detected in the image, the second packetincluding: a header including information regarding the line; and apayload including region data concerning a partial region correspondingto the line, out of the region,

wherein the image processing unit recognizes whether or not the payloadof the second packet includes the region data, based on at least one ofinformation included in the header corresponding to the payload or theregion information.

(22)

A communication system comprising:

a transmission device including a first image processing unit thatcontrols to transmit a first packet including region informationregarding a region detected from an image and controls to transmit asecond packet for each of lines in which at least the region has beendetected in the image, the second packet including a header includinginformation regarding the line, and a payload including region dataconcerning a partial region corresponding to the line, out of theregion; and

a reception device including a second image processing unit thatrestores a partial image corresponding to the region on the basis of aresult of reception of a first packet and a result of reception of asecond packet for each of lines in which at least the region has beendetected in the image,

wherein the second image processing unit recognizes whether or not thepayload of the second packet includes the region data, based on at leastone of information included in the header corresponding to the payloador the region information.

REFERENCE SIGNS LIST

-   -   1000 COMMUNICATION SYSTEM    -   100 IMAGE SENSOR    -   102 IMAGE SENSOR DEVICE    -   104, 130, 300 IC CHIP    -   106, 132, 302 IMAGE PROCESSING UNIT    -   108, 304, 334 LINK/PHY CONTROLLER    -   110, 306, 336 CONTROL CODE GENERATOR    -   112, 134, 308, 338 PH GENERATOR    -   114, 138, 310, 340 EBD BUFFER    -   116, 312, 342 IMAGE DATA BUFFER    -   118, 136, 314 COMBINING UNIT    -   120, 328, 358 TRANSMITTING UNIT    -   122, 316, 346 REGION CUTOUT UNIT    -   124, 326 IMAGE PROCESSING CONTROLLER    -   126, 140, 324, 354 ENCODER    -   318, 348 REGION ANALYZER    -   320, 350 OVERLAP DETECTOR    -   322, 352 PRIORITY SETTING UNIT    -   200 PROCESSOR    -   202, 232, 402, 432 HEADER SEPARATOR    -   204, 234, 404, 434 HEADER INTERPRETER    -   206, 236, 406, 436 PAYLOAD SEPARATOR    -   208, 238, 408, 438 EBD INTERPRETER    -   210 FRAME MEMORY    -   212, 240, 412, 442 IMAGE PROCESSING UNIT    -   214, 242, 414, 444 INFORMATION EXTRACTION UNIT    -   216, 244, 416, 446 REGION DECODER    -   218, 246, 418, 448 REGION IMAGE GENERATOR    -   220, 420, 450 ORDINARY IMAGE DECODER    -   410, 440 REGION DATA SEPARATOR    -   800 MEMORY    -   900 DISPLAY DEVICE

The invention claimed is:
 1. A transmission device comprising an imageprocessing unit that controls to transmit a first packet includingregion information regarding a region detected from an image andcontrols to transmit a second packet for each of lines in which at leastthe region has been detected in the image, the second packet including:a header including information regarding the line; and a payloadincluding region data concerning a partial region corresponding to theline, out of the region, wherein the header includes: identificationinformation of the data included in the payload; an error correctioncode of the information included in the header; and an extension regionprovided so as to be interposed between the identification informationand the error correction code, and information indicating whether or notthe region data is included in the payload corresponding to the headeris set in at least a part of the extension region.
 2. The transmissiondevice according to claim 1, wherein the header includes: informationindicating a start of a frame, information indicating an end of theframe, information indicating whether or not a corresponding line isvalid, a line number, information indicating whether or not a line hasembedded data, the identification information, the extension region, andthe error correction code, being arranged in this order.
 3. Thetransmission device according to claim 1, wherein the second packet isindividually transmitted for each of the partial regions included in theline.
 4. The transmission device according to claim 3, wherein thepayload includes information regarding coordinates of the correspondingpartial region, and information indicating that the payload includes theinformation regarding the coordinates is set in the extension region. 5.The transmission device according to claim 1, wherein the second packetis transmitted for each of lines, and the payload included in the secondpacket includes the region data for each of the partial regions includedin the line.
 6. The transmission device according to claim 5, whereinthe payload of the second packet corresponding to the line including aplurality of the partial regions includes data corresponding to aninterval between two partial regions separated from each other in theimage, between the region data corresponding to each of the two partialregions.
 7. The transmission device according to claim 6, wherein thepayload includes information regarding coordinates of a top partialregion out of the partial regions included in the corresponding line,and information indicating that the payload includes informationregarding the coordinates is set in the extension region.
 8. Thetransmission device according to claim 5, wherein a position, within theimage, of the partial region corresponding to each of one or more piecesof the region data included in the payload is specified based on theregion information regarding the region including the partial regionincluded in the first packet.
 9. The transmission device according toclaim 5, wherein the payload includes information regarding the partialregion included in the corresponding line, and information indicatingtype of information regarding the partial region included in the payloadis set in the extension region.
 10. The transmission device according toclaim 9, wherein the information regarding the partial region includesinformation regarding the number of the partial regions included in thecorresponding line and includes at least one of identificationinformation, coordinates, or length regarding the partial region foreach of the partial regions.
 11. The transmission device according toclaim 9, wherein, in the payload corresponding to a second lineimmediately succeeding a first line, information having a set valuedifferent from the first line out of the information regarding thepartial region is set.
 12. The transmission device according to claim 1,wherein the region information includes at least one of identificationinformation, coordinates, height, width, AD word length, exposure, gain,which are related to the region, information regarding a subject, orinformation for post-stage signal processing.
 13. A reception devicecomprising an image processing unit that restores a partial imagecorresponding to a region detected from an image, based on a result ofreception of a first packet including region information regarding theregion and based on a result of reception of a second packet for each oflines in which at least the region has been detected in the image, thesecond packet including: a header including information regarding theline; and a payload including region data concerning a partial regioncorresponding to the line, out of the region, wherein the headerincludes: identification information of the data included in thepayload; an error correction code of the information included in theheader; and an extension region provided so as to be interposed betweenthe identification information and the error correction code, andinformation indicating whether or not the region data is included in thepayload corresponding to the header is set in at least a part of theextension region.
 14. The reception device according to claim 13,wherein, in a case where information indicating that the region data isincluded in the payload corresponding to the header is set for at leasta part of the extension region of the header, the image processing unitrestores the partial image corresponding to the region, based on theresult of reception of the first packet and the result of reception ofthe second packet.
 15. The reception device according to claim 13,wherein the image processing unit restores the partial imagecorresponding to the region by performing restoration processing on theregion data included in the payload of one or more of the secondpackets, based on the region information.
 16. The reception deviceaccording to claim 13, wherein the second packet is transmittedindividually for each of the partial regions included in the line, thepayload includes information regarding coordinates of the correspondingpartial region, information indicating that the payload includes theinformation regarding the coordinates is set in the extension region,and the image processing unit performs restoration processing on theregion data included in the payload of the one or more second packetsand thereby restores the partial image corresponding to the region, foreach of the regions set in the image, based on the region information,and the information regarding the coordinates included in the payload ofthe one or more second packets corresponding to the partial region ofthe region.
 17. The reception device according to claim 13, wherein thepayload of the second packet corresponding to the line including aplurality of the partial regions includes data corresponding to aninterval between two partial regions separated from each other in theimage, between the region data corresponding to each of the two partialregions, the payload includes information regarding coordinates of a toppartial region out of the partial regions included in the correspondingline, Information indicating that the payload includes the informationregarding coordinates is set in the extension region, and the imageprocessing unit performs, for each of the regions, restorationprocessing on the region data included in the payload of the one or moresecond packets and thereby restores the partial image corresponding tothe region, based on the region information, and the coordinatesincluded in the payload of the one or more second packets.
 18. Thereception device according to claim 13, wherein the payload includesinformation regarding the partial region included in the correspondingline, information indicating the type of information regarding thepartial region included in the payload is set in the extension region,and the image processing unit performs, for each of the regions,restoration processing on the region data included in the payload of theone or more second packets and thereby restores the partial imagecorresponding to the region, based on the region information and theinformation regarding the partial region included in the payload whichhas been extracted on the basis of the information indicating the typeset in the extension region for each of the one or more second packets.19. A communication system comprising: a transmission device including afirst image processing unit that controls to transmit a first packetincluding region information regarding a region detected from an imageand controls to transmit a second packet for each of lines in which atleast the region has been detected in the image, the second packetincluding a header including information regarding the line, and apayload including region data concerning a partial region correspondingto the line, out of the region; and a reception device including asecond image processing unit that restores a partial image correspondingto the region on the basis of a result of reception of the first packetand a result of reception of the second packet for each of lines inwhich at least the region has been detected in the image, wherein theheader includes: identification information of the data included in thepayload; an error correction code of the information included in theheader; and an extension region provided so as to be interposed betweenthe identification information and the error correction code, andinformation indicating whether or not the region data is included in thepayload corresponding to the header is set in at least a part of theextension region.
 20. A transmission device comprising an imageprocessing unit that controls to transmit a first packet includingregion information regarding a region detected from an image andcontrols to transmit a second packet for each of lines in which at leastthe region has been detected in the image, the second packet including:a header including information regarding the line; and a payloadincluding region data concerning a partial region corresponding to theline, out of the region, wherein whether or not the payload of thesecond packet includes the region data is recognized on the receivingside of the second packet, based on at least one of information includedin the header corresponding to the payload or the region information.21. A reception device comprising an image processing unit that restoresa partial image corresponding to a region detected from an image, basedon a result of reception of a first packet including region informationregarding the region and based on a result of reception of a secondpacket for each of lines in which at least the region has been detectedin the image, the second packet including: a header includinginformation regarding the line; and a payload including region dataconcerning a partial region corresponding to the line, out of theregion, wherein the image processing unit recognizes whether or not thepayload of the second packet includes the region data, based on at leastone of information included in the header corresponding to the payloador the region information.
 22. A communication system comprising: atransmission device including a first image processing unit thatcontrols to transmit a first packet including region informationregarding a region detected from an image and controls to transmit asecond packet for each of lines in which at least the region has beendetected in the image, the second packet including a header includinginformation regarding the line, and a payload including region dataconcerning a partial region corresponding to the line, out of theregion; and a reception device including a second image processing unitthat restores a partial image corresponding to the region on the basisof a result of reception of a first packet and a result of reception ofa second packet for each of lines in which at least the region has beendetected in the image, wherein the second image processing unitrecognizes whether or not the payload of the second packet includes theregion data, based on at least one of information included in the headercorresponding to the payload or the region information.